According to the recently introduced theory of artistic support tools, creativity support tools exert normative influences over artistic production, instantiating a normative ground that shapes both the process and product of artistic expression.We argue that the normative ground of most existing automated writing tools is misaligned with writerly values and identify a potential alternative frame-material writing support-for experimental poetry tools that flexibly support the finding, processing, transforming, and shaping of text(s).Based on this frame, we introduce Phraselette, an artistic material writing support interface that helps experimental poets search for words and phrases.To provide material writing support, Phraselette is designed to counter the dominant mode of automated writing tools, while offering language model affordances in line with writerly values.We further report on an extended expert evaluation involving 10 published poets that indicates support for both our framing of material writing support and for Phraselette itself.

Figure 1: Fuzzy linkography allows for the rapid translation of user activity logs from digital creativity support tools (and other traces of creative activity) into rough graphical summaries, suitable for visual and quantitative inspection by researchers.

We introduce Toyteller, an AI-powered storytelling system where users generate a mix of story text and visuals by directly manipulating character symbols like they are toy-playing. Anthropomorphized symbol motions can convey rich and nuanced social interactions; Toyteller leverages these motions (1) to let users steer story text generation and (2) as a visual output format that accompanies story text. We enabled motion-steered text generation and text-steered motion generation by mapping motions and text onto a shared semantic space so that large language models and motion generation models can use it as a translational layer. Technical evaluations showed that Toyteller outperforms a competitive baseline, GPT-4o. Our user study identified that toy-playing helps express intentions difficult to verbalize. However, only motions could not express all user intentions, suggesting combining it with other modalities like language. We discuss the design space of toy-playing interactions and implications for technical HCI research on human-AI interaction.

Generative AI has established the opportunity to readily transform content from one medium to another.This capability is especially powerful for storytelling, where visual illustrations can illuminate a story originally expressed in text.In this paper, we focus on the task of narrative scene illustration, which involves automatically generating an image depicting a scene in a story.Motivated by recent progress on textto-image models, we consider a pipeline that uses LLMs as an interface for prompting textto-image models to generate scene illustrations given raw story text.We apply variations of this pipeline to a prominent story corpus in order to synthesize illustrations for scenes in these stories.We conduct a human annotation task to obtain pairwise quality judgments for these illustrations.The outcome of this process is the SCENEILLUSTRATIONS dataset, which we release as a new resource for future work on crossmodal narrative transformation.Through our analysis of this dataset and experiments modeling illustration quality, we demonstrate that LLMs can effectively verbalize scene knowledge implicitly evoked by story text.Moreover, this capability is impactful for generating and evaluating illustrations.

Creating an abridged version of a text involves shortening it while maintaining its linguistic qualities.In this paper, we examine this task from an NLP perspective for the first time.We present a new resource, ABLIT, which is derived from abridged versions of English literature books.The dataset captures passage-level alignments between the original and abridged texts.We characterize the linguistic relations of these alignments, and create automated models to predict these relations as well as to generate abridgements for new texts.Our findings establish abridgement as a challenging task, motivating future resources and research.The dataset is available at github.com/roemmele/AbLit.

This thesis explores the use of a recurrent neural network model for a novel story generation task. In this task, the model analyzes an ongoing story and generates a sentence that continues the story.

People naturally anthropomorphize the movement of nonliving objects, as social psychologists Fritz Heider and Marianne Simmel demonstrated in their influential 1944 research study. When they asked participants to narrate an animated film of two triangles and a circle moving in and around a box, participants described the shapes' movement in terms of human actions. Using a framework for authoring and annotating animations in the style of Heider and Simmel, we established new crowdsourced datasets where the motion trajectories of animated shapes are labeled according to the actions they depict. We applied two machine learning approaches, a spatial-temporal bag-of-words model and a recurrent neural network, to the task of automatically recognizing actions in these datasets. Our best results outperformed a majority baseline and showed similarity to human performance, which encourages further use of these datasets for modeling perception from motion trajectories. Future progress on simulating human-like motion perception will require models that integrate motion information with top-down contextual knowledge.

Humans have a remarkable tendency to anthropomorphize moving objects, ascribing to them intentions and emotions as if they were human. Early social psychology research demonstrated that animated film clips depicting the movements of simple geometric shapes could elicit rich interpretations of intentional behavior from viewers. In attempting to model this reasoning process in software, we first address the problem of automatically recognizing humanlike actions in the trajectories of moving shapes. There are two main difficulties. First, there is no defined vocabulary of actions that are recognizable to people from motion trajectories. Second, in order for an automated system to learn actions from motion trajectories using machine-learning techniques, a vast amount of these action- trajectory pairs is needed as training data. This paper describes an approach to data collection that resolves both of these problems. In a web-based game, called Triangle Charades, players create motion trajectories for actions by animating a triangle to depict those actions. Other players view these animations and guess the action they depict. An action is considered recognizable if players can correctly guess it from animations. To move towards defining a controlled vocabulary and collecting a large dataset, we conducted a pilot study in which 87 users played Triangle Charades. Based on this data, we computed a simple metric for action recognizability. Scores on this metric formed a gradual linear pattern, suggesting there is no clear cutoff for determining if an action is recognizable from motion data. These initial results demonstrate the advantages of using a game to collect data for this action recognition task.