In the future of work discourse, AI is touted as the ultimate productivity amplifier. Yet, beneath the efficiency gains lie subtle erosions of human expertise and agency. This paper shifts focus from the future of work to the future of workers by navigating the AI-as-Amplifier Paradox: AI’s dual role as enhancer and eroder, simultaneously strengthening performance while eroding underlying expertise. We present a year-long study on the longitudinal use of AI in a high-stakes workplace among cancer specialists. Initial operational gains hid “intuition rust”: the gradual dulling of expert judgment. These asymptomatic effects evolved into chronic harms, such as skill atrophy and identity commoditization. Building on these findings, we offer a framework for dignified Human-AI interaction co-constructed with professional knowledge workers facing AI-induced skill erosion without traditional labor protections. The framework operationalizes sociotechnical immunity through dual-purpose mechanisms that serve institutional quality goals while building worker power to detect, contain, and recover from skill erosion, and preserve human identity. Evaluated across healthcare and software engineering, our work takes a foundational step toward dignified human-AI interaction futures by balancing productivity with the preservation of human expertise.

World models, which encapsulate the dynamics of how actions affect environments, are foundational to the functioning of intelligent agents. In this work, we explore the potential of Large Language Models (LLMs) to operate as world models. Although LLMs are not inherently designed to model real-world dynamics, we show that they can be induced to perform two critical world model functions: determining the applicability of an action based on a given world state, and predicting the resulting world state upon action execution. This is achieved by fine-tuning two separate LLMs—one for precondition prediction and another for effect prediction—while leveraging synthetic data generation techniques. Through human-participant studies, we validate that the precondition and effect knowledge generated by our models aligns with human understanding of world dynamics. We also analyze the extent to which the world model trained on our synthetic data results in an inferred state space that supports the creation of action chains, a necessary property for planning.

Reinforcement learning (RL) using world models has found significant recent successes. However, when a sudden change to world mechanics or properties occurs then agent performance and reliability can dramatically decline. We refer to the sudden change in visual properties or state transitions as novelties. Implementing novelty detection within generated world model frameworks is a crucial task for protecting the agent when deployed. In this paper, we propose straightforward bounding approaches to incorporate novelty detection into world model RL agents by utilizing the misalignment of the world model’s hallucinated states and the true observed states as a novelty score. We provide effective approaches to detecting novelties in a distribution of transitions learned by an agent in a world model. Finally, we show the advantage of our work in a novel environment compared to traditional machine learning novelty detection methods as well as currently accepted RL-focused novelty detection algorithms.

Abstract Reinforcement learning (RL) systems can be complex and non-interpretable, making it challenging for non-AI experts to understand or intervene in their decisions. This is due in part to the sequential nature of RL in which actions are chosen because of their likelihood of obtaining future rewards. However, RL agents discard the qualitative features of their training, making it difficult to recover user-understandable information for “why” an action is chosen. We propose a technique Experiential Explanations to generate counterfactual explanations by training influence predictors along with the RL policy. Influence predictors are models that learn how different sources of reward affect the agent in different states, thus restoring information about how the policy reflects the environment. Two human evaluation studies revealed that participants presented with Experiential Explanations were better able to correctly guess what an agent would do than those presented with other standard types of explanation. Participants also found that Experiential Explanations are more understandable, satisfying, complete, useful, and accurate. Qualitative analysis provides information on the factors of Experiential Explanations that are most useful and the desired characteristics that participants seek from the explanations.

Explainability of AI systems is critical for users to take informed actions. Understanding who opens the black-box of AI is just as important as opening it. We conduct a mixed-methods study of how two different groups—people with and without AI background—perceive different types of AI explanations. Quantitatively, we share user perceptions along five dimensions. Qualitatively, we describe how AI background can influence interpretations, elucidating the differences through lenses of appropriation and cognitive heuristics. We find that (1) both groups showed unwarranted faith in numbers for different reasons and (2) each group found value in different explanations beyond their intended design. Carrying critical implications for the field of XAI, our findings showcase how AI generated explanations can have negative consequences despite best intentions and how that could lead to harmful manipulation of trust. We propose design interventions to mitigate them.

Automated story generation has been one of the long-standing challenges in NLP.Among all dimensions of stories, suspense is very common in human-written stories but relatively underexplored in AI-generated stories.While recent advances in large language models (LLMs) have greatly promoted language generation in general, state-of-the-art LLMs are still unreliable when it comes to suspenseful story generation.We propose a novel iterative-promptingbased planning method that is grounded in two theoretical foundations of story suspense from cognitive psychology and narratology.This theory-grounded method works in a fully zeroshot manner and does not rely on any supervised story corpora.To the best of our knowledge, this paper is the first attempt at suspenseful story generation with LLMs.Extensive human evaluations of the generated suspenseful stories demonstrate the effectiveness of our method.

Kaige Xie, Tong Yu, Haoliang Wang, Junda Wu, Handong Zhao, Ruiyi Zhang, Kanak Mahadik, Ani Nenkova, Mark Riedl. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Reinforcement Learning with Human Feedback (RLHF) is the most prominent method for Language Model (LM) alignment. However, RLHF is an unstable and data-hungry process that continually requires new high-quality LM-generated data for finetuning. We introduce Advantage-Leftover Lunch RL (A-LoL), a new class of offline policy gradient algorithms that enable RL training on any pre-existing data. By assuming the entire LM output sequence as a single action, A-LoL allows incorporating sequence-level classifiers or human-designed scoring functions as rewards. Subsequently, by using LM’s value estimate, A-LoL only trains on positive advantage (leftover) data points, making it resilient to noise. Overall, A-LoL is an easy-to-implement, sample-efficient, and stable LM training recipe. We demonstrate the effectiveness of A-LoL and its variants with a set of four different language generation tasks. We compare against both online RL (PPO) and recent preference-based (DPO, PRO) and reward-based (GOLD) offline RL baselines. On the commonly-used RLHF benchmark, Helpful and Harmless Assistant (HHA), LMs trained with A-LoL methods achieve the highest diversity while also being rated more safe and helpful than the baselines according to humans. Additionally, in the remaining three tasks, A-LoL could optimize multiple distinct reward functions even when using noisy or suboptimal training data.

Values or principles are key elements of human society that influence people to behave and function according to an accepted standard set of social rules to maintain social order. As AI systems are becoming ubiquitous in human society, it is a major concern that they could violate these norms or values and potentially cause harm. Thus, to prevent intentional or unintentional harm, AI systems are expected to take actions that align with these principles. Training systems to exhibit this type of behavior is difficult and often requires a specialized dataset. This work presents a multi-modal dataset illustrating normative and non-normative behavior in real-life situations described through natural language and artistic images. This training set contains curated sets of images that are designed to teach young children about social principles. We argue that this is an ideal dataset to use for training socially normative agents given this fact.

Mistakes in AI systems are inevitable, arising from both technical limitations and sociotechnical gaps. While black-boxing AI systems can make the user experience seamless, hiding the seams risks disempowering users to mitigate fallouts from AI mistakes. Instead of hiding these AI imperfections, can we leverage them to help the user? While Explainable AI (XAI) has predominantly tackled algorithmic opaqueness, we propose that seamful design can foster AI explainability by revealing and leveraging sociotechnical and infrastructural mismatches. We introduce the concept of Seamful XAI by (1) conceptually transferring "seams" to the AI context and (2) developing a design process that helps stakeholders anticipate and design with seams. We explore this process with 43 AI practitioners and real end-users, using a scenario-based co-design activity informed by real-world use cases. We found that the Seamful XAI design process helped users foresee AI harms, identify underlying reasons (seams), locate them in the AI's lifecycle, learn how to leverage seamful information to improve XAI and user agency. We share empirical insights, implications, and reflections on how this process can help practitioners anticipate and craft seams in AI, how seamfulness can improve explainability, empower end-users, and facilitate Responsible AI.

Explainable AI (XAI) systems are sociotechnical in nature; thus, they are subject to the sociotechnical gap-divide between the technical affordances and the social needs. However, charting this gap is challenging. In the context of XAI, we argue that charting the gap improves our problem understanding, which can reflexively provide actionable insights to improve explainability. Utilizing two case studies in distinct domains, we empirically derive a framework that facilitates systematic charting of the sociotechnical gap by connecting AI guidelines in the context of XAI and elucidating how to use them to address the gap. We apply the framework to a third case in a new domain, showcasing its affordances. Finally, we discuss conceptual implications of the framework, share practical considerations in its operationalization, and offer guidance on transferring it to new contexts. By making conceptual and practical contributions to understanding the sociotechnical gap in XAI, the framework expands the XAI design space.

We teach goal-driven agents to interactively act and speak in situated environments by training on generated curriculums.Our agents operate in LIGHT ( Urbanek et al., 2019)-a large-scale crowd-sourced fantasy text adventure game wherein an agent perceives and interacts with the world through textual natural language.Goals in this environment take the form of character-based quests, consisting of personas and motivations.We augment LIGHT by learning to procedurally generate additional novel textual worlds and quests to create a curriculum of steadily increasing difficulty for training agents to achieve such goals.In particular, we measure curriculum difficulty in terms of the rarity of the quest in the original training distribution-an easier environment is one that is more likely to have been found in the unaugmented dataset.An ablation study shows that this method of learning from the tail of a distribution results in significantly higher generalization abilities as measured by zeroshot performance on never-before-seen quests.

Sarah Wiegreffe, Jack Hessel, Swabha Swayamdipta, Mark Riedl, Yejin Choi. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.

We focus on the task of creating a reinforcement learning agent that is inherently explainable---with the ability to produce immediate local explanations by thinking out loud while performing a task and analyzing entire trajectories post-hoc to produce temporally extended explanations. This Hierarchically Explainable Reinforcement Learning agent (HEX-RL), operates in Interactive Fictions, text-based game environments in which an agent perceives and acts upon the world using textual natural language. These games are usually structured as puzzles or quests with long-term dependencies in which an agent must complete a sequence of actions to succeed---providing ideal environments in which to test an agent's ability to explain its actions. Our agent is designed to treat explainability as a first-class citizen, using an extracted symbolic knowledge graph-based state representation coupled with a Hierarchical Graph Attention mechanism that points to the facts in the internal graph representation that most influenced the choice of actions. Experiments show that this agent provides significantly improved explanations over strong baselines, as rated by human participants generally unfamiliar with the environment, while also matching state-of-the-art task performance.

Automated game design is the problem of automatically producing games through computational processes. Traditionally, these methods have relied on the authoring of search spaces by a designer, defining the space of all possible games for the system to the author. In this article, we instead learn representations of existing games from gameplay video and use these to approximate a search space of novel games. In a human subject study, we demonstrate that these novel games are indistinguishable from human games in terms of challenge and that one of the novel games was equivalent to one of the human games in terms of fun, frustration, and likeability.

Automated story plot generation is the task of generating a coherent sequence of plot events. Causal relations between plot events are believed to increase the perception of story and plot coherence. In this work, we introduce the concept of soft causal relations as causal relations inferred from commonsense reasoning. We demonstrate C2PO, an approach to narrative generation that operationalizes this concept through Causal, Commonsense Plot Ordering. Using human-participant protocols, we evaluate our system against baseline systems with different commonsense reasoning reasoning and inductive biases to determine the role of soft causal relations in perceived story quality. Through these studies we also probe the interplay of how changes in commonsense norms across storytelling genres affect perceptions of story quality.

As AI-powered systems increasingly mediate consequential decision-making, their explainability is critical for end-users to take informed and accountable actions. Explanations in human-human interactions are socially-situated. AI systems are often socio-organizationally embedded. However, Explainable AI (XAI) approaches have been predominantly algorithm-centered. We take a developmental step towards socially-situated XAI by introducing and exploring Social Transparency (ST), a sociotechnically informed perspective that incorporates the socio-organizational context into explaining AI-mediated decision-making. To explore ST conceptually, we conducted interviews with 29 AI users and practitioners grounded in a speculative design scenario. We suggested constitutive design elements of ST and developed a conceptual framework to unpack ST’s effect and implications at the technical, decision-making, and organizational level. The framework showcases how ST can potentially calibrate trust in AI, improve decision-making, facilitate organizational collective actions, and cultivate holistic explainability. Our work contributes to the discourse of Human-Centered XAI by expanding the design space of XAI.

Dialogue models trained on human conversations inadvertently learn to generate toxic responses.In addition to producing explicitly offensive utterances, these models can also implicitly insult a group or individual by aligning themselves with an offensive statement.To better understand the dynamics of contextually offensive language, we investigate the stance of dialogue model responses in offensive Reddit conversations.Specifically, we create TOXICHAT, a crowd-annotated dataset of 2,000 Reddit threads and model responses labeled with offensive language and stance.Our analysis reveals that 42% of human responses agree with toxic comments, whereas only 13% agree with safe comments.This undesirable behavior is learned by neural dialogue models, such as DialoGPT, which we show are two times more likely to agree with offensive comments.To enable automatic detection of offensive language, we fine-tuned transformerbased classifiers on TOXICHAT that achieve 0.71 F 1 for offensive labels and 0.53 Macro-F 1 for stance labels.Finally, we quantify the effectiveness of controllable text generation (CTG) methods to mitigate the tendency of neural dialogue models to agree with offensive comments.Compared to the baseline, our best CTG model achieves a 19% reduction in agreement with offensive comments and produces 29% fewer offensive replies.Our work highlights the need for further efforts to characterize and analyze inappropriate behavior in dialogue models, in order to help make them safer. 1

World models improve a learning agent's ability to efficiently operate in interactive and situated environments. This work focuses on the task of building world models of text-based game environments. Text-based games, or interactive narratives, are reinforcement learning environments in which agents perceive and interact with the world using textual natural language. These environments contain long, multi-step puzzles or quests woven through a world that is filled with hundreds of characters, locations, and objects. Our world model learns to simultaneously: (1) predict changes in the world caused by an agent's actions when representing the world as a knowledge graph; and (2) generate the set of contextually relevant natural language actions required to operate in the world. We frame this task as a Set of Sequences generation problem by exploiting the inherent structure of knowledge graphs and actions and introduce both a transformer-based multi-task architecture and a loss function to train it. A zero-shot ablation study on never-before-seen textual worlds shows that our methodology significantly outperforms existing textual world modeling techniques as well as the importance of each of our contributions.

This paper explores character-driven story continuation, in which the story emerges through characters' first-and second-person narration as well as dialogue-requiring models to select language that is consistent with a character's persona and their relationships with other characters while following and advancing the story.We hypothesize that a multi-task model that trains on character dialogue plus character relationship information improves transformer-based story continuation.To this end, we extend the Critical Role Dungeons and Dragons Dataset (Rameshkumar and Bailey, 2020)-consisting of dialogue transcripts of people collaboratively telling a story while playing the role-playing game Dungeons and Dragons-with automatically extracted relationships between each pair of interacting characters as well as their personas.A series of ablations lend evidence to our hypothesis, showing that our multi-task model using character relationships improves story continuation accuracy over strong baselines.

Neural network based approaches to automated story plot generation attempt to learn how to generate novel plots from a corpus of natural language plot summaries. Prior work has shown that a semantic abstraction of sentences called events improves neural plot generation and and allows one to decompose the problem into: (1) the generation of a sequence of events (event-to-event) and (2) the transformation of these events into natural language sentences (event-to-sentence). However, typical neural language generation approaches to event-to-sentence can ignore the event details and produce grammatically-correct but semantically-unrelated sentences. We present an ensemble-based model that generates natural language guided by events. We provide results—including a human subjects study—for a full end-to-end automated story generation system showing that our method generates more coherent and plausible stories than baseline approaches 1.

Value alignment is a property of an intelligent agent indicating that it can only pursue goals and activities that are beneficial to humans. Traditional approaches to value alignment use imitation learning or preference learning to infer the values of humans by observing their behavior. We introduce a complementary technique in which a value-aligned prior is learned from naturally occurring stories which encode societal norms. Training data is sourced from the children's educational comic strip, Goofus & Gallant. In this work, we train multiple machine learning models to classify natural language descriptions of situations found in the comic strip as normative or non-normative by identifying if they align with the main characters' behavior. We also report the models' performance when transferring to two unrelated tasks with little to no additional training on the new task.

Large-scale, transformer-based language models such as GPT-2 are pretrained on diverse corpora scraped from the internet.Consequently, they are prone to generating non-normative text (i.e. in violation of social norms).We introduce a technique for fine-tuning GPT-2, using a policy gradient reinforcement learning technique and a normative text classifier to produce reward and punishment values.We evaluate our technique on five data sets using automated and human participant experiments.The normative text classifier is 81-90% accurate when compared to gold-standard human judgements of normative and non-normative generated text.Our normative fine-tuning technique is able to reduce non-normative text by 27-61%, depending on the data set.

Machine learning advances have afforded an increase in algorithms capable of creating art, music, stories, games, and more. However, it is not yet well-understood how machine learning algorithms might best collaborate with people to support creative expression. To investigate how practicing designers perceive the role of AI in the creative process, we developed a game level design tool for Super Mario Bros.-style games with a built-in AI level designer. In this paper we discuss our design of the Morai Maker intelligent tool through two mixed-methods studies with a total of over one-hundred participants. Our findings are as follows: (1) level designers vary in their desired interactions with, and role of, the AI, (2) the AI prompted the level designers to alter their design practices, and (3) the level designers perceived the AI as having potential value in their design practice, varying based on their desired role for the AI.

The ability to extract sequences of game events for high-resolution e-sport games has traditionally required access to the game's engine. This serves as a barrier to groups who don't possess this access. It is possible to apply deep learning to derive these logs from gameplay video, but it requires computational power that serves as an additional barrier. These groups would benefit from access to these logs, such as small e-sport tournament organizers who could better visualize gameplay to inform both audience and commentators. In this paper we present a combined solution to reduce the required computational resources and time to apply a convolutional neural network (CNN) to extract events from e-sport gameplay videos. This solution consists of techniques to train a CNN faster and methods to execute predictions more quickly. This expands the types of machines capable of training and running these models, which in turn extends access to extracting game logs with this approach. We evaluate the approaches in the domain of DOTA2, one of the most popular e-sports. Our results demonstrate our approach outperforms standard backpropagation baselines.

Language-modeling--based approaches to story plot generation attempt to construct a plot by sampling from a language model (LM) to predict the next character, word, or sentence to add to the story. LM techniques lack the ability to receive guidance from the user to achieve a specific goal, resulting in stories that don't have a clear sense of progression and lack coherence. We present a reward-shaping technique that analyzes a story corpus and produces intermediate rewards that are backpropagated into a pre-trained LM in order to guide the model toward a given goal. Automated evaluations show our technique can create a model that generates story plots which consistently achieve a specified goal. Human-subject studies show that the generated stories have more plausible event ordering than baseline plot generation techniques.

Automated rationale generation is an approach for real-time explanation generation whereby a computational model learns to translate an autonomous agent's internal state and action data representations into natural language. Training on human explanation data can enable agents to learn to generate human-like explanations for their behavior. In this paper, using the context of an agent that plays Frogger, we describe (a) how to collect a corpus of explanations, (b) how to train a neural rationale generator to produce different styles of rationales, and (c) how people perceive these rationales. We conducted two user studies. The first study establishes the plausibility of each type of generated rationale and situates their user perceptions along the dimensions of confidence, humanlike-ness, adequate justification, and understandability. The second study further explores user preferences between the generated rationales with regard to confidence in the autonomous agent, communicating failure and unexpected behavior. Overall, we find alignment between the intended differences in features of the generated rationales and the perceived differences by users. Moreover, context permitting, participants preferred detailed rationales to form a stable mental model of the agent's behavior.

Designers structure a game to provide a desired range of player behaviors: the play space of a game. Any given game is one instance from a design space of alternatives. Navigating a design space to achieve a designer's goals requires knowledge of how design choices shape the play space in a game. We present algorithms to automatically measure play patterns using statistical models that predict how design choices alter player behavior. We present Monte Carlo tree search as a way to sample behaviors from a play space, action metrics to automate play space measurement, and predictive modeling techniques to model design spaces. We demonstrate these techniques in two simplified, perfect information, adversarial game domains based on Scrabble and Hearthstone showing their use for automated design space modeling.

Automated story generation is the problem of automatically selecting a sequence of events, actions, or words that can be told as a story. We seek to develop a system that can generate stories by learning everything it needs to know from textual story corpora. To date, recurrent neural networks that learn language models at character, word, or sentence levels have had little success generating coherent stories. We explore the question of event representations that provide a mid-level of abstraction between words and sentences in order to retain the semantic information of the original data while minimizing event sparsity. We present a technique for preprocessing textual story data into event sequences. We then present a technique for automated story generation whereby we decompose the problem into the generation of successive events (event2event) and the generation of natural language sentences from events (event2sentence). We give empirical results comparing different event representations and their effects on event successor generation and the translation of events to natural language.

We introduce \em AI rationalization, an approach for generating explanations of autonomous system behavior as if a human had performed the behavior. We describe a rationalization technique that uses neural machine translation to translate internal state-action representations of an autonomous agent into natural language. We evaluate our technique in the Frogger game environment, training an autonomous game playing agent to rationalize its action choices using natural language. A natural language training corpus is collected from human players thinking out loud as they play the game. We motivate the use of rationalization as an approach to explanation generation and show the results of two experiments evaluating the effectiveness of rationalization. Results of these evaluations show that neural machine translation is able to accurately generate rationalizations that describe agent behavior, and that rationalizations are more satisfying to humans than other alternative methods of explanation.

We present a novel approach to player modeling based on a convolutional neural net trained on game event logs. We test our approach and a hybrid extension over two distinct games, a clone of Super Mario Bros. and Gwario, a human computation version of Super Mario Bros.: The Lost Levels. We demonstrate high accuracy in predicting a variety of measures of player experience across these two games. Further we present evidence that our technique derives quality design knowledge and demonstrate the ability to build a more general model.

Human computation games (HCGs) can provide novel solutions to intractable computational problems, help enable scientific breakthroughs, and provide datasets for artificial intelligence. However, our knowledge about how to design and deploy HCGs that appeal to players and solve problems effectively is incomplete. We present an investigatory HCG based on Super Mario Bros. We used this game in a human subjects study to investigate how different social conditions---singleplayer and multiplayer---and scoring mechanics---collaborative and competitive---affect players' subjective experiences, accuracy at the task, and the completion rate. In doing so, we demonstrate a novel design approach for HCGs, and discuss the benefits and tradeoffs of these mechanics in HCG design.

Human computation games (HCGs) are a crowdsourcing approach to solving computationally-intractable tasks using games. We outline a formal representation of the mechanics in HCGs, providing a structural breakdown to visualize, compare, and explore the space of HCG mechanics. We present a methodology based on small-scale design experiments using fixed tasks while varying game elements to observe effects on both the player experience and the human computation task completion. Ultimately, we wish enable easier exploration and development of HCGs, letting these games provide meaningful experiences to players while solving difficult problems.

Intelligent agents need to be able to make predictions about their environment. In this work we present a novel approach to learn a forward simulation model via simple search over pixel input. We make use of a video game, Super Mario Bros., as an initial test of our approach as it represents a physics system that is significantly less complex than reality. We demonstrate the significant improvement of our approach in predicting future states compared with a baseline CNN and apply the learned model to train a game playing agent. Thus we evaluate the algorithm in terms of the accuracy and value of its output model.

Interactive narrative is a form of storytelling in which users affect a dramatic storyline through actions by assuming the role of characters in a virtual world.This extended abstract outlines the Scheherazade-IF system, which uses crowdsourcing and artificial intelligence to automatically construct text-based interactive narrative experiences.

We present an exploratory analysis of the effects of game world topologies on self-reported player experience in Computer Role Playing Games (CRPGs). We find that (a) players are more engaged in game worlds that better match their self-reported preferences; and (b) player preferences for game topology can be predicted based on their in-game behavior. We further describe how in-game behavioral features that correlate to preferences can be used to control procedural content generation algorithms.

Digital games often center on a series of challenges designed to vary in difficulty over the course of the game. Designers, however, lack ways to ensure challenges are suitably tailored to the abilities of each game player, often resulting in player boredom or frustration. Challenge tailoring refers to the general problem of matching designer-intended challenges to player abilities. We present an approach to predict temporal player performance and select appropriate content to solve the challenge tailoring problem. Our temporal collaborative filtering approach-tensor factorization-captures similarities among players and the challenges they face to predict player performance on unseen, future challenges. Tensor factorization accounts for varying player abilities over time and is a generic approach capable of modeling many kinds of players. We use constraint solving to optimize content selection to match player skills to a designer-specified level of performance and present a model-performance curves-for designers to specify desired, temporally changing player behavior. We evaluate our approach in a role-playing game through two empirical studies of humans and one study using simulated agents. Our studies show tensor factorization scales in multiple game-relevant data dimensions, can be used for modestly effective game adaptation, and can predict divergent player learning trends.

We introduce Dramatis, a computational model of suspense based on a reformulation of a psychological definition of the suspense phenomenon. In this reformulation, suspense is correlated with the audience’s ability to generate a plan for the protagonist to avoid an impending negative outcome. Dramatis measures the suspense level by generating such a plan and determining its perceived likelihood of success. We report on three evaluations of Dramatis, including a comparison of Dramatis output to the suspense reported by human readers, as well as ablative tests of Dramatis components. In these studies, we found that Dramatis output corresponded to the suspense ratings given by human readers for stories in three separate domains.

Game designs often center on the game mechanics - rules governing the logical evolution of the game. We seek to develop an intelligent system that generates computer games. As first steps towards this goal we present a composable and cross-domain representation for game mechanics that draws from AI planning action representations. We use a constraint solver to generate mechanics subject to design requirements on the form of those mechanics - what they do in the game. A planner takes a set of generated mechanics and tests whether those mechanics meet playability requirements - controlling how mechanics function in a game to affect player behavior. We demonstrate our system by modeling and generating mechanics in a role-playing game, platformer game, and combined role-playing-platformer game.

Games With a Purpose can enable an intelligent agent to persistently and pervasively sense the real world by using game players as reconfigurable sensors. We propose a technique whereby an intelligent agent incentivizes players to collect data by translating data collection tasks into a series of quests played on a mobile device. In this paper, we define the concept of Proactive Sensing and provide a framework for Game-Based Proactive Sensing that can adapt games and narrative that optimizes for data collection and long-term player engagement.

In story-based games or other interactive systems, a drama manager (DM) is an omniscient agent that acts to bring about a particular sequence of plot points for the player to experience. Traditionally, the DM's narrative evaluation criteria are solely derived from a human designer. We present a DM that learns a model of the player's storytelling preferences and automatically recommends a narrative experience that is predicted to optimize the player's experience while conforming to the human designer's storytelling intentions. Our DM is also capable of manipulating the space of narrative trajectories such that the player is more likely to make choices that result in the recommended experience. Our DM uses a novel algorithm, called prefix-based collaborative filtering (PBCF), that solves the sequential recommendation problem to find a sequence of plot points that maximizes the player's rating of his or her experience. We evaluate our DM in an interactive storytelling environment based on choose-your-own-adventure novels. Our experiments show that our algorithms can improve the player's experience over the designer's storytelling intentions alone and can deliver more personalized experiences than other interactive narrative systems while preserving players' agency.

Story generation is the problem of automatically selecting a sequence of events that meet a set of criteria and can be told as a story. Story generation is knowledge-intensive; traditional story generators rely on a priori defined domain models about fictional worlds, including characters, places, and actions that can be performed. Manually authoring the domain models is costly and thus not scalable. We present a novel class of story generation system that can generate stories in an unknown domain. Our system (a) automatically learns a domain model by crowdsourcing a corpus of narrative examples and (b) generates stories by sampling from the space defined by the domain model. A large-scale evaluation shows that stories generated by our system for a previously unknown topic are comparable in quality to simple stories authored by untrained humans

Machinima is a new form of creative digital filmmaking that leverages the real time graphics rendering of computer game engines. Because of the low barrier to entry, machinima has become a popular creative medium for hobbyists and novices while still retaining borrowed conventions from professional filmmaking. Can novice machinima creators benefit from creativity support tools? A preliminary study shows novices generally have difficulty adhering to cinematographic conventions. We identify and document four cinematic conventions novices typically violate. We report on a Wizard-of-Oz study showing a rule-based intelligent system that can reduce the frequency of errors that novices make by providing information about rule violations without prescribing solutions. We discuss the role of error reduction in creativity support tools.

Game Artificial Intelligence (Game AI) is a sub-discipline of Artificial Intelligence (AI) and Machine Learning (ML) that explores the ways in which AI and ML can augment player experiences in computer games. Storytelling is an integral part of many modern computer games; within games stories create context, motivate the player, and move the action forward. Interactive Narrative is the use of AI to create and manage stories within games, creating the perception that the player is a character in a dynamically unfolding and responsive story. This paper introduces Game AI and focuses on the open research problems of Interactive Narrative.

Scenario-based training exemplifies the learning-by-doing approach to human performance improvement. In this paper, we enumerate the advantages of incorporating automated scenario generation technologies into the traditional scenario development pipeline. An automated scenario generator is a system that creates training scenarios from scratch, augmenting human authoring to rapidly develop new scenarios, providing a richer diversity of tailored training opportunities, and delivering training scenarios on demand. We introduce a combinatorial optimization approach to scenario generation to deliver the requisite diversity and quality of scenarios while tailoring the scenarios to a particular learner's needs and abilities. We propose a set of evaluation metrics appropriate to scenario generation technologies and present preliminary evidence for the suitability of our approach compared to other scenario generation approaches.

This paper reports on an empirical study that uses a Grounded Theory approach to investigate the creative practices of Machinima filmmakers. Machinima is a new digital film production technique that uses the 3D graphics and real time rendering capability of video game engines to create films. In contrast to practices used in traditional film production, we've found that Machinima filmmakers explore and evaluate ideas in real time. These filmmakers generate vague and underspecified mental images, which are then explored and refined using the real time rendering capabilities of game engines. The game engine assists the filmmaker to fill in indeterminate details, which allows creative exploration of scenes through playfully experimenting with parameters such as camera angle and position, lighting, and character position. Creative exploration distributes the cognitive task of evaluation between the human user and the Machinima tool to enable evaluation through exploring possible scene configurations.

Science-fiction and fantasy stories often contain objects never envisioned previously. Inventing gadgets like lightsabers or mythical creatures like griffins is a creative task. Traditional computational storytelling systems are limited in their expressivity because they cannot create new types of objects or gadgets. The Japanese manga series Doraemon exemplifies the role of new and creative gadgets in creating fun and successful stories. We surveyed five volumes of Doraemon and identified 9 cognitive strategies of gadget creation, unified in a 5-step process. We present an algorithm to create new types of gadgets in the context of story generation. The algorithm is a combination of partial-order planning and analogical reasoning. Although Doraemon is our motivating example, we can also generate gadgets commonly seen in other science fictions and fairy tales.

We address the problem of building computational agents that are capable of play. Existing research has examined the forms, characteristics, and processes involved in various kinds of play at a high level. However, this research does not provide a unified framework at a level of detail sufficient for building computational agents that can play. As a step toward addressing this gap we synthesize diverse research on pretend play to recognize important components of pretend play agents. We also develop a formal model of one key component of pretend play, pretend object play, and present a computational implementation of this model. Our work provides initial criteria for the content and processes necessary for pretend play agents.

Alternate Reality Games layer a fictional world over the real world in order to provide players with a location-based interactive narrative experience. Building off previous work on game flow and enjoyment metrics in games, we present a metric based on the key elements that empirical studies suggest make for enjoyable ARG gameplay. We empirically validate our metric and call out key elements of ARGs that are most likely to have bearing on the success of a game.

Narrative, and in particular storytelling, is an important part of the human experience. Consequently, computational systems that can reason about narrative can be more effective communicators, entertainers, educators, and trainers. One of the central challenges in computational narrative reasoning is narrative generation, the automated creation of meaningful event sequences. There are many factors -- logical and aesthetic -- that contribute to the success of a narrative artifact. Central to this success is its understandability. We argue that the following two attributes of narratives are universal: (a) the logical causal progression of plot, and (b) character believability. Character believability is the perception by the audience that the actions performed by characters do not negatively impact the audience's suspension of disbelief. Specifically, characters must be perceived by the audience to be intentional agents. In this article, we explore the use of refinement search as a technique for solving the narrative generation problem -- to find a sound and believable sequence of character actions that transforms an initial world state into a world state in which goal propositions hold. We describe a novel refinement search planning algorithm -- the Intent-based Partial Order Causal Link (IPOCL) planner -- that, in addition to creating causally sound plot progression, reasons about character intentionality by identifying possible character goals that explain their actions and creating plan structures that explain why those characters commit to their goals. We present the results of an empirical evaluation that demonstrates that narrative plans generated by the IPOCL algorithm support audience comprehension of character intentions better than plans generated by conventional partial-order planners.

This paper presents a methodology for automatically customizing a scenario to suit a learner's abilities, needs, or goals. Training scenarios are often utilized to give learners hands-on experience with real-life problem solving tasks. The customization of scenarios has the potential to improve learning gains within these domains. We present initial steps toward an intelligent technology called a Scenario Adaptor that employs a partial order planning formalism to reason about learning objectives and causality, and we discuss how the Scenario Adaptor may add or delete learning objectives from a scenario.

This paper presents preliminary findings from our empirical study of the cognition employed by performers in improvisational theatre. Our study has been conducted in a laboratory setting with local improvisers. Participants performed predesigned improv "games", which were videotaped and shown to each individual participant for a retrospective protocol collection. The participants were then shown the video again as a group to elicit data on group dynamics, misunderstandings, etc. This paper presents our initial findings that we have built based on our initial analysis of the data and highlights details of interest.

Human creativity plays an important role in the production of many of the media products that permeate our society. However, non-expert creators are often limited by a lack of technical ability, as opposed to creative ability. This is especially true for story authoring. We present an approach to supporting creativity using synthetic audience - an intelligent agent that acts as (a) a surrogate story recipient and (b) critic capable of providing constructive feedback. We describe initial efforts based on computational modeling of cognitive processes and creativity.

The creation of novel, engaging and dynamic interactive stories presents a unique challenge to the designers of systems for interactive entertainment, education and training. Unlike conventional narrative media, an interactive narrative-based system may be required to generate its own story structure, determine the appropriate interface elements to use to convey the storys action and manage the effective interaction of a user within the story as it plays out. Here we describe the architecture of the Mimesis system, which integrates a 3D graphical gaming environment with intelligent techniques for generating and controlling interaction with and within a narrative in order to create an engaging and coherent user experience

The ease with which a user interface can be navigated strongly contributes to its usability. In this paper we describe preliminary results of a project aimed at making the evaluation of user interfaces from this perspective more routine. We have designed a system to carry out an autonomous, exploratory navigation through the graphical user interface of interactive, off-the-shelf software applications. The system is not a robust tool, but rather a proof of concept that can exhibit interesting behaviors. The traversal process generates a representation of the connectivity of the user interface, as well as navigational paths to specific commands. The reasoning component of the system is based on the ACT-R architecture, while the perceptual and motor components of the system are built on top of the SegMan perception/action substrate. We present the design of the system and its use in exploring a simple user interface.

This paper describes a set of visualization techniques for interactive planning in a physical force simulation called AFS. We have developed a 3D environment in which textures are overlaid on a simulated landscape to convey information about environmental properties, agent actions, and possible strategies. Scenes are presented, via automated camera planning, such that some simple agent goals can be induced visually with little effort. These two areas of visualization functionality in AFS exploit properties of human low-level and intermediate-level vision, respectively. This paper presents AFS, its visualization environment, and studies we have run to explore the relationship between AFS visualizations and the high-level planning process.

Social navigation is the process of making navigational decisions in real or virtual environments based on social and communicative interaction with others. A computational model for social navigation is presented as an extension to an existing framework for general navigation, reducing decision-making to the minimization of cognitive costs. Consideration for social navigation gives rise to a classification framework based on the synchronicity, directness, and social presence during social interaction, each of which has direct effect on the cognitive costs of navigational tasks. Finally, a new recommender system, TRAILGUIDE, is presented as a tool that facilitates social navigation by allowing authors to explicitly publish "trails" within and between World Wide Web pages.