Abstract Much research in human–artificial intelligence (AI) interaction treats cooperation as a design problem of alignment, asking how system behavior can be coordinated with human intentions, task goals or expectations in order to support effective collaborative action. In this debate contribution, we shift the analytic focus from alignment to sequential organisation and ask how cooperation is interactionally accomplished when one source of action does not demonstrably orient to a shared sense of reality, yet its outputs are nevertheless incorporated into an unfolding sequence of action. Drawing on a sequential comic-making practice involving a generative AI system, we analyse three examples to examine how creative work can progress through misrecognition, breakdown and material effects, not necessarily requiring consensus or control. We argue that such misalignment is not a failure to be repaired but can function as a recurring interactional condition in human–AI cooperation, sustained through the sequential uptake, reinterpretation and transformation of AI-generated outputs. Based on this analysis, we suggest that human–AI interaction and design research would benefit from more explicitly interactional accounts of cooperation that treat misalignment as a productive condition rather than something to be minimised. We position artistic creative practice as a methodologically revealing site for examining how agency, authorship and coordination are practically accomplished in cooperative work with AI and we discuss implications for human-centered design.

A common challenge for people on the autism spectrum with intellectual disability, is indoor navigation and related daily activities, as found in previous research. In this paper we report on co-design of assistive augmented reality applications, where the goal is to help people on the autism spectrum to gain more independence in their daily lives. This study is based on initially two full-day workshops with staff only, followed by ten individual workshops with the end-users and their staff at day centers, using a mix of methods and prototypes. The results show a clear potential of augmented reality as assistive technology for indoor navigation, depending on individual capability and/or complexity of environments, as well as for other activities. We also found that new barriers may arise, which are discussed for future research.

Some people with a motor disability that limits hand movements use technology to draw via their eyes. Free-eye drawing has been re-investigated recently and yielded state-of-the-art results via unimodal gaze control. However, limitations remain, including limited functions, conflicts between observation and drawing, and the brush tailing issue. We introduce a professional unimodal gaze control free-eye drawing application and improve upon free-eye drawing by extended gaze-based user interface functions, improved brush dynamics, and a double-blink gaze gesture. An experiment and a field study were conducted to assess the system’s usability compared to the mainstream gaze-control drawing method and hand drawing and the accessibility among users with motor disabilities. The results showed that the application provides efficient interaction and the ability to create hand-sketch-level graphics for people with motor disabilities. Herein, we contribute a robust and professional free-eye drawing application, detailing valuable design considerations for future developments in gaze interaction.

EyeCompass is a novel free-eye drawing system enabling high-fidelity and efficient free-eye drawing through unimodal gaze control, addressing the bottlenecks of gaze-control drawing. EyeCompass helps people to draw using only their eyes, which is of value to people with motor disabilities. Currently, there is no effective gaze-control drawing application due to multiple challenges including involuntary eye movements, conflicts between visuomotor transformation and ocular observation, gaze trajectory control, and inherent eye-tracking errors. EyeCompass addresses this using two initial gaze-control drawing mechanisms: brush damping dynamics and the gaze-oriented method. The user experiments compare the existing gaze-control drawing method and EyeCompass, showing significant improvements in the drawing performance of the mechanisms concerned. The field study conducted with motor-disabled people produced various creative graphics and indicates good usability of the system. Our studies indicate that EyeCompass is a high-fidelity, accurate, feasible free-eye drawing method for creating artistic works via unimodal gaze control.

Synthesizing realistic clouds is a complex and demanding task, as clouds are characterized by random shapes, complex scattering and turbulent appearances. Existing approaches either employ two-dimensional image matting or three-dimensional physical simulations. This paper proposes a novel sketch-to-image deep learning system using fast sketches to paint and edit volumetric clouds. We composed a dataset of 2000 real cloud images and translated simple strokes into authentic clouds based on a conditional generative adversarial network (cGAN). Compared to previous cloud simulation methods, our system demonstrates more efficient and straightforward processes to generate authentic clouds for computer graphics, providing a widely accessible sky scene design approach for use by novices, amateurs, and expert artists.

The papers in this special section focus on user experiences of artificial intelligence in games.

Tabletop roleplaying games (TTRPGs) and procedural content generators can both be understood as systems of rules for producing content. In this paper, we argue that TTRPG design can usefully be viewed as procedural content generator design. We present several case studies linking key concepts from PCG research – including possibility spaces, expressive range analysis, and generative pipelines – to key concepts in TTRPG design. We then discuss the implications of these relationships and suggest directions for future work uniting research in TTRPGs and PCG.

Tarot has been used for centuries as a method to give structure to storytelling, both in game and fortune-telling settings. As such, tarot cards have developed over time, expanding the symbolism and depth of meaning associated with each card. This provides a corpus for a large number of possible stories, making tarot a rich area of exploration for story generation. Therefore, we have created a tarot-based narrative generation system that creates short movie-like story synopses, along with a tagline one might see on a movie poster. This project is in early development; we have created a prototype as a proof of concept. The project exists as a webpage that an interactor can use to draw new tarot cards for the story spread (card layout) and generate new stories from them. In this paper we discuss the details of our system and describe more details about the tarot as a corpus. We also discuss future areas of exploration based on our proof of concept.

This paper describes the Mind Module (MM) which can be used in game worlds to assign avatars and autonomous characters personality traits, emotions, emotional attachments and moods which together form finely granulated states which are especially appropriate in systems where individualized play experiences are desired. The MM is implemented as a spreading activation network where the different nodes represent affective properties and has been used in several experimental game prototypes. In this paper, results are reported from play-tests where representations of affective processes were used as game-play elements. The MM API can be used with the particular nodes described in this paper or extended for use with nodes appropriate to other applications.