Large-scale prompt-based AI generators have broadened access to media creation but often introduce friction in sustained creative practice. Visual artists must navigate limited personalization, narrow cultural coverage, homogenized aesthetics, and broader concerns around sustainability, bias, and data-scraping ethics. Small data and model crafting offer alternatives that center artists’ agency over data and models. We examine this approach through Autolume, a no-code, locally run tool for crafting and navigating generative models in real time. Across five focus groups with 25 visual artists in three two-day workshops, Autolume was used as a research probe to explore small-data practices. Findings show that participants viewed small data as a personal and responsible alternative to large-scale AI, but identified model training as a technical barrier, underscoring the need for improved explainability and support. We highlight design opportunities for AI art tools, including low-level controls, mapping UIs to ML processes, ecosystem integration, and small data as a pedagogical tool.

Foreign language speaking anxiety (FLSA) poses a major challenge for English-language learners, suppressing confidence and triggering a cycle of avoidance that hinders language acquisition. To address this, we explored the use of LLM-based embodied conversational agents (ECA) in social virtual reality (VR), which provide personalized support and multimodal interaction in a contextualized environment. We developed three English-language learning scenarios in social VR and conducted a five-day mixed-methods study where participants (N=20) engaged in daily 30-minute role-play practice with an LLM-based ECA to evaluate the efficacy of the system. Quantitative results showed a significant reduction in self-reported FLAS after 3 days, along with subtle gains in speaking proficiency measures. Qualitatively, learners perceived increased confidence, attributing it to the LLM-based ECA’s non-judgmental stance, linguistic scaffolding, affective encouragement, and adaptive feedback. Our findings suggest the potential of LLM-based ECAs in social VR for language learning and offer considerations for future agent design.

We present and release MIDI-GPT, a generative system based on the Transformer architecture that is designed for computer-assisted music composition workflows. MIDI-GPT supports the infilling of musical material at the track and bar level, and can condition generation on attributes including: instrument type, musical style, note density, polyphony level, and note duration. In order to integrate these features, we employ an alternative representation for musical material, creating a time-ordered sequence of musical events for each track and concatenating several tracks into a single sequence, rather than using a single time-ordered sequence where the musical events corresponding to different tracks are interleaved. We also propose a variation of our representation allowing for expressiveness. We present experimental results that demonstrate that MIDI-GPT is able to consistently avoid duplicating the musical material it was trained on, generate music that is stylistically similar to the training dataset, and that attribute controls allow enforcing various constraints on the generated material. We also outline several real-world applications of MIDI-GPT, including collaborations with industry partners that explore the integration and evaluation of MIDI-GPT into commercial products, as well as several artistic works produced using it.

We present the Predictive Gameplay-based Layered Affect Model (PreGLAM), an affective game spectator model that flexibly integrates into a game design process. PreGLAM combines elements of real-time Player Experience Models, and Affective Non-Player-Character models to output real-time estimated values for a spectator's valence, arousal, and tension during gameplay. Because tension is related to prospective events, PreGLAM attempts to predict future gameplay events. We implement and evaluate PreGLAM in a custom gameGalactic Defense, which we also describe. PreGLAM significantly outperforms a random walk time series in how accurately it matches ground-truth annotations, and has comparable accuracy to state of the art affect models.

With the rise of artificial intelligence (AI), there has been increasing interest in human-AI co-creation in a variety of artistic domains including music as AI-driven systems are frequently able to generate human-competitive artifacts. Now, the implications of such systems for the musical practice are being investigated. This paper reports on a thorough evaluation of the user adoption of the Multi-Track Music Machine (MMM) as a minimal co-creative AI tool for music composers. To do this, we integrate MMM into Cubase, a popular Digital Audio Workstation (DAW), by producing a "1-parameter" plugin interface named MMM-Cubase, which enables human-AI co-composition. We conduct a 3-part mixed method study measuring usability, user experience and technology acceptance of the system across two groups of expert-level composers: hobbyists and professionals. Results show positive usability and acceptance scores. Users report experiences of novelty, surprise and ease of use from using the system, and limitations on controllability and predictability of the interface when generating music. Findings indicate no significant difference between the two user groups.

Calliope is a web application for co-creative multi-track music composition (MMM) in the symbolic domain. It is built to facilitate the use of multi-track music machine (MMM). The user can upload Musical Instrument Digital Interface (MIDI) files, visualize and edit MIDI tracks, and generate partial (via bar in-filling) or complete multi-track content using the Multi-Track Music Machine (MMM). Generation of new MIDI excerpts can be done in batch and can be combined with active playback listening for an enhanced Computer-assisted Composition (CAC) workflow. The user can export generated materials as MIDI files or directly stream MIDI playback from the system to their favorite Digital Audio Workstation (DAW). Calliope can be used for creative ideation and for exploring alternatives of musical phrases in composition.

We present and evaluate an application of affective adaptive generative music in a single-player, action-RPG video game. We create a score that serves as an audience to the gameplay, based on the output of PreGLAM, which models the emotional perception of a game audience. We use the Multi-track Music Machine to expand and extend a composed adaptive musical score, and we use industry-standard production techniques to synthesize and perform all of our musical scores. We evaluate our application of generative music in comparison to two composed scores, one adaptive and one linear. Our generative score is rated as nearly equivalent to a composed linear score in perceptions of emotional congruency, immersion, and preference.

Bio-responsive immersive Virtual Reality can transform our interactions to bring awareness to our physiological rhythms fostering connection with our bodies, each other and nature. JeL is an immersive installation that aims to foster a feeling of connection through the process of breathing synchronization. Two immersants synchronize their breathing to fuel the growth of a coral-like structure that, together with the interactions of others, populates an initially empty coral reef. JeL is designed to support an intimate connection between users and with nature, sending a message about our collective capacity to care for the environment. JeL is an installation and research platform for exploring breathing synchronization and its effect on the feeling of connection. It was well received at a digital art festival where participants were able to relax and synchronize using the installation. Reflection on our design process and observations provides insights for the development of systems that promote connection.

Integrating physical learning materials with mobile device applications may have benefits for early childhood learning. We present three techniques for creating a hybrid tangible-augmented reality (T-AR) enabling technology platform. This platform enables researchers to develop applications that use readily available physical learning materials, such as letters, numbers, symbols or shapes. The techniques are visual marker-based; computer-vision and machine-learning; and capacitive touches. We describe details of implementation and demonstrate these techniques through a use case of a reading tablet app that uses wooden/plastic letters for input and augmented output. Our comparative analysis revealed that the machine-learning technique most flexibly sensed different physical letter sets but had variable accuracy impacted by lighting and tracking lag at this time. Lastly, we demonstrate how this enabling technology can be generalized to a variety of early learning apps through a second use case with physical numbers.

Third wave HCI initiated a slow transformation in the methods of UX research: from widely used quantitative approaches to more recently employed qualitative techniques. Articulating the nuances, complexity, and diversity of a user's experience beyond surface descriptions remains a challenge within design. One qualitative method — micro-phenomenology — has been used in HCI/Design research since 2001. Yet, no systematic understanding of micro-phenomenology has been presented, particularly from the perspective of HCI/Design researchers who actively use it in design contexts. We interviewed 5 HCI/Design experts who utilize micro-phenomenology and present their experiences with the method. We illustrate how this method has been applied by the selected experts through developing a practice, and present conditions under which the descriptions of the experience unfold, and the values that this method can provide to HCI/Design field. Our contribution highlights the value of micro-phenomenology in articulating the experience of designers and participants, developing vocabulary for multi-sensory experiences, and unfolding embodied tacit knowledge.

Attending to breath is a self-awareness practice that exists within many contemplative and reflective traditions and is recognized for its benefits to well-being. Our current technological landscape embraces a large body of systems that utilize breath data in order to foster self-awareness. This paper seeks to deepen our understanding of the design space of systems that perceptually extend breath awareness. Our contribution is twofold: (1) our analysis reveals how the underlying theoretical frameworks shape the system design and its evaluation, and (2) how system design features support perceptual extension of breath awareness. We review and critically analyze 31 breath-based interactive systems. We identify 4 theoretical frameworks and 3 design strategies for interactive systems that perceptually extend breath awareness. We reflect upon this design space from both a theoretical and system design perspective, and propose future design directions for developing systems that "listen to" breath and perceptually extend it.

Realistic recordings of soundscapes often have multiple sound events co-occurring, such as car horns, engine and human voices. Sound event retrieval is a type of contentbased search aiming at finding audio samples, similar to an audio query based on their acoustic or semantic content. State of the art sound event retrieval models have focused on single-label audio recordings, with only one sound event occurring, rather than on multi-label audio recordings (i.e., multiple sound events occur in one recording). To address this latter problem, we propose different Deep Learning architectures with a Siamesestructure and a Pairwise Presence Matrix. The networks are trained and evaluated using the SONYC-UST dataset containing both single- and multi-label soundscape recordings. The performance results show the effectiveness of our proposed model.

Whether literally or suggestively, the concept of soundscape is alluded in both modern and ancient music. In this study, we examine whether we can analyze and compare Western and Chinese classical music based on soundscape models. We addressed this question through a comparative study. Specifically, corpora of Western classical music excerpts (WCMED) and Chinese classical music excerpts (CCMED) were curated and annotated with emotional valence and arousal through a crowdsourcing experiment. We used a sound event detection (SED) and soundscape emotion recognition (SER) models with transfer learning to predict the perceived emotion of WCMED and CCMED. The results show that both SER and SED models could be used to analyze Chinese and Western classical music. The fact that SER and SED work better on Chinese classical music emotion recognition provides evidence that certain similarities exist between Chinese classical music and soundscape recordings, which permits transferability between machine learning models.

We present Taksim, an Answer Set Programming (ASP) framework for generating content in games through constrained graph partitioning. We illustrate its expressivity by implementing logical constraints that are relevant to generating the spaces of game levels. Furthermore, we present a case study for creating game levels from a given Mission Graph. Finally, we propose key concepts that make constrained graph partitioning, coupled with ASP, an asset for Procedural Content Generation.

Music is an important affective aspect of video games. We present the findings of an empirical study on the affective effects of adaptive uses of music in games. We find that adaptive music can significantly increase a players reported experienced feeling of tension, that players recognize and value music, and that player recognize and value adaptive music over linear music.

Busy daily lives and ongoing distractions often make people feel disconnected from their bodies and experiences. Guided attention to self can alleviate this disconnect as in focused-attention meditation, in which breathing often constitutes the primary object on which to focus attention. In this context, sustained breath awareness plays a crucial role in the emergence of the meditation experience. We designed an immersive virtual environment (iVE) with a generative soundtrack that supports sustained attention on breathing by employing the users' breathing in interaction. Both sounds and visuals are directly mapped to the user's breathing patterns, thus bringing the awareness researched. We conducted micro-phenomenology interviews to unfold the process in which breath awareness can be induced and sustained in this environment. The findings revealed the mechanisms by which audio and visual cues in VR can elicit and foster breath-awareness, and unfolded the nuances of this process through subjective experiences of the study participants. Finally, the results emphasize the important role that a sense of agency and control have in shaping the overall quality of the experience. This can in turn inform the design specifications of future mindfulness-based designs focused on breath awareness.

Soundscape emotion recognition (SER) aims at the automatic recognition of emotions perceived in soundscape recordings. To benchmark SER, we propose a dataset of audio samples called Emo-Soundscapes and two evaluation protocols for machine learning models. We curated 600 soundscape recordings from Freesound.org and mixed 613 audio clips from a combination of these. The Emo-Soundscapes dataset contains 1213 6-second Creative Commons licensed audio clips. We collected the ground truth annotations of perceived emotion in these 1213 soundscape recordings using a crowdsourcing listening experiment, where 1182 annotators from 74 different countries rank the audio clips according to the perceived valence and arousal. This dataset allows studying SER and how the mixing of various soundscape recordings influences their perceived emotion. The dataset is at http://metacreation.net/emo-soundscapes/.

A music video (MV) is a videotaped performance of a recorded popular song, usually accompanied by dancing and visual images. In this paper, we outline the design of a generative music video system, which automatically generates an audio-video mashup for a given target audio track. The system performs segmentation for the given target song based on beat detection. Next, according to audio similarity analysis and color heuristic selection methods, we obtain generated video segments. Then, these video segments are truncated to match the length of audio segments and are concatenated as the final music video. An evaluation of our system has shown that users are receptive to this novel presentation of music videos and are interested in future developments.

In order to facilitate interaction in computer-mediated communication and enrich user experience in general, we introduce a novel textual emotion visualization approach, grounded in generative art and evocative visuals. The approach is centered on the idea that affective computer systems should be able to relate to, communicate, and evoke human emotions. It maps emotions identified in the text to evocative abstract animation. We examined two visualizations based on our approach and two common textual emotion visualization techniques, chat emoticons and avatars, along three dimensions: emotion communication, emotion evocation, and overall user enjoyment. Our study, organized as repeated measures within-subject experiment, demonstrated that in terms of emotion communication, our visualizations are comparable with emoticons and avatars. However, our main visualization based on abstract color, motion, and shape proved to be the best in evoking emotions. In addition, in terms of the overall user enjoyment, it gave results comparable with emoticons, but better than avatars.

The expressivity of virtual, animated agents plays an important role in their believability. While the planning and goal-oriented aspects of agent movements have been addressed in the literature extensively, expressing the emotional state of the agents in their movements is an open research problem. We present our interactive animated agent model with controllable affective movements. We have recorded a corpus of affect-expressive motion capture data of two actors, performing various movements, and annotated based on their arousal and valence levels. We train a Factored, Conditional Restricted Boltzmann Machine (FCRBM) with this corpus in order to capture and control the valence and arousal qualities of movement patterns. The agents are then able to control the emotional qualities of their movements through the FCRBM for any given combination of the valence and arousal. Our results show that the model is capable of controlling the arousal level of the synthesized movements, and to some extent their valence, through manually defining the level of valence and arousal of the agent, as well as making transitions from one state to the other. We validate the expressive abilities of the model through conducting an experiment where participants were asked to rate their perceived affective state for both the generated and recorded movements.

While single-accelerometers are a common consumer embedded sensors, their use in representing movement data as an intelligent resource remains scarce. Accelerometers have been used in movement recognition systems, but rarely to assess expressive qualities of movement. We present a prototype of wearable system for the real-time detection and classification of movement quality using acceleration data. The system applies Laban Movement Analysis (LMA) to recognize Laban Effort qualities from acceleration input using a Machine Learning software that generates classifications in real time. Existing LMA-recognition systems rely on motion capture data and video data, and can only be deployed in controlled settings. Our single-accelerometer system is portable and can be used under a wide range of environmental conditions. We evaluate the performance of the system, present two applications using the system in the digital arts and discuss future directions.

A sound synthesizer can be defined as a program that takes a few input parameters and returns a sound. The general sound synthesis problem could then be formulated as: given a sound (or a set of sounds) what program and set of input parameters can generate that sound (set of sounds)? We propose a novel approach to tackle this problem in which we represent sound synthesizers using Pure Data (Pd), a graphic programming language for digital signal processing. We search the space of possible sound synthesizers using Coevolutionary Mixed-typed Cartesian Genetic Programming (MT-CGP), and the set of input parameters using a standard Genetic Algorithm (GA). The proposed algorithm co-evolves a population of MT-CGP graphs, representing the functional forms of synthesizers, and a population of GA chromosomes, representing their inputs parameters. A fitness function based on the Mel-frequency Cepstral Coefficients (MFCC) evaluates the distance between the target and produced sounds. Our approach is capable of suggesting novel functional forms and input parameters, suitable to approximate a given target sound (and we hope in future iterations a set of sounds). Since the resulting synthesizers are presented as Pd patches, the user can experiment, interact with, and reuse them.

"Dreaming Machine #3" (Prototype 2) is the second iteration of work in progress towards the "Dreaming Machine #3" (DM3), the third in a series of site-specific generative art installations informed by conceptions of dreaming. DM3 senses its visual environment through a static video camera where images are segmented by perceptual processes. Segmented percepts are clustered and serve as the material from which dreams, generative and free-associative compositions, are constructed.

It has been suggested that creativity can be functionally segregated into two processes: spontaneous and deliberate. In this paper, we propose that the spontaneous aspect of creativity is enabled by the same neural simulation mechanisms that have been implicated in visual mentation (e.g. visual perception, mental imagery, mind-wandering and dreaming). This proposal is developed into an Integrative Theory that serves as the foundation for a computational model of dreaming and site-specific artwork: A Machine that Dreams.

We present GESMI (Generative Electronica Statistical Modeling Instrument), a software system that generates Electronic Dance Music (EDM) using evolutionary methods. While using machine learning, GESMI rests on a corpus analysed and transcribed by domain experts. We describe a method for generating the overall form of a piece and individual parts, including specific patterns sequences, using evolutionary algorithms. Lastly, we describe how the user can use contextually-relevant target features to query the generated database of strong individual patterns. As our main focus is upon artistic results, our methods themselves use an iterative, somewhat evolutionary, design process based upon our reaction to results.

Online human textual interaction often carries important emotional meanings inaccessible to computers. We propose an approach to textual emotion recognition in the context of computer-mediated communication. The proposed recognition approach works at the sentence level and uses the standard Ekman emotion classification. It is grounded in a refined keyword-spotting method that employs: a WordNet-based word lexicon, a lexicon of emoticons, common abbreviations and colloquialisms, and a set of heuristic rules. The approach is implemented through the Synesketch software system. Synesketch is published as a free, open source software library. Several Synesketch-based applications presented in the paper, such as the the emotional visual chat, stress the practical value of the approach. Finally, the evaluation of the proposed emotion recognition algorithm shows high accuracy and promising results for future research and applications.

The Level Generation Competition, part of the IEEE Computational Intelligence Society (CIS)-sponsored 2010 Mario AI Championship, was to our knowledge the world's first procedural content generation competition. Competitors participated by submitting level generators - software that generates new levels for a version of Super Mario Bros tailored to individual players' playing style. This paper presents the rules of the competition, the software used, the scoring procedure, the submitted level generators, and the results of the competition. We also discuss what can be learned from this competition, both about organizing procedural content generation competitions and about automatically generating levels for platform games. The paper is coauthored by the organizers of the competition (the first three authors) and the competitors.

This paper presents an approach to automatic video game level design consisting of a computational model of player enjoyment and a generative system based on evolutionary computing. The model estimates the entertainment value of game levels according to the presence of “rhythm groups,” which are defined as alternating periods of high and low challenge. The generative system represents a novel combination of genetic algorithms (GAs) and constraint satisfaction (CS) methods and uses the model as a fitness function for the generation of fun levels for two different games. This top-down approach improves upon typical bottom-up techniques in providing semantically meaningful parameters such as difficulty and player skill, in giving human designers considerable control over the output of the generative system, and in offering the ability to create levels for different types of games.

Heterogenesis is an interactive sound and tactile installation consisting of a group of autonomous artificial agents that collectively generate and evolve a soundscape in response to one another and to the presence of human participants. Taking the form of long pillar-like sculptures and forming a rudimentary artificial neural network, the agents respond to their environment in three ways: (1) by altering their generated sounds, (2) by communicating with one another so as to “warn” of human presence nearby, and (3) by producing an inaudible acoustic pressure field that “pushes” participants as they approach. Heterogenesis represents an attempt to manifest complex emergent behavior through a rich set of interactions with autonomous systems - both human and machine.

We address the problem of multi-robot area coverage and present a new approach in the case where the map of the area and its static obstacles are known and the robots have a limited visibility range. The proposed method starts by locating a set of static guards on the map of the target area and then builds a graph called Reduced-CDT, a new environment representation method based on theConstrained Delaunay Triangulation(CDT).Multi-Prim'sis used to decompose the graph into a forest ofpartial spanning trees(PSTs). Each PST is then modified through a mechanism calledConstrained Spanning Tour(CST) to build a cycle which is then assigned to an individual robot. Subsequently, robots start navigating the cycles and consequently cover the whole area. We show that the proposed approach is complete and robust with respect to robot failure.

Eavesdropping is an internet-based, interactive audio system that explores network mediated, musical performance in shared public spaces. The project aims to develop an environment which increases audience interaction and connectedness in a localized, computer-controlled performance. The system is a client-server architecture made of three components: (1) an audio preparation interface, (2) an interactive performance interface, and (3) a machine learning-based conductor. An artificial conductor mixes an acoustic ecology based on mood data entered by participants while learning from their feedback. Technicalities and early evaluation are presented.

While argumentation-based negotiation has been accepted as a promising alternative to game-theoretic or heuristic based negotiation, no evidence has been provided to confirm this theoretical advantage. We propose a model of bilateral negotiation extending a simple monotonic concession protocol by allowing the agents to exchange information about their underlying interests and possible alternatives to achieve them during the negotiation. We present an empirical study that demonstrates (through simulation) the advantages of this interest-based negotiation approach over the more classic monotonic concession approach to negotiation.