Detailed measurements of piano key motion capture touch, timing, and dynamic control, providing crucial performance insights. Such expressive gestures are overlooked in MIDI, which only records pitch onset, duration, and velocity. Here, we introduce a novel transcription technique that directly maps audio from expressive piano performance to continuous piano key motion. User studies reveal a preference to the transcribed key motion trajectories over MIDI in representing sound, and over 80% accuracy in matching transcribed trajectories to audio from contrasting piano expressions. Follow-up interviews further indicate that the visualised trajectories can reveal subtle performance nuances and provide actionable guidance for both teaching and practice. An interface example for pedagogy and performance analysis utilising our technique is also illustrated. By providing a physically grounded performance representation that musicians can interpret and act upon, this work establishes a foundation for future interactive tools in music pedagogy, performance feedback, and embodied musical learning.

Auditory working memory is essential for various daily activities, such as language acquisition and conversation. It involves the temporary storage and manipulation of information that is no longer present in the environment. While extensively studied in neuroscience and cognitive science, research on its modeling within neural networks remains limited. To address this gap, we propose a general framework based on a closed-loop predictive coding paradigm to perform short auditory signal memory tasks. The framework is evaluated on two widely used benchmark datasets for environmental sound and speech, demonstrating high semantic similarity across both datasets.

Today’s deep learning architectures are primarily based on perceptron models, which do not capture the oscillatory dynamics characteristic of biological neurons. Although oscillatory systems have recently gained attention for their closer resemblance to neural behavior, they still fall short of modeling the intricate spatio-temporal interactions observed in natural neural circuits. In this paper, we propose a bio-inspired oscillatory state system (BioOSS) designed to emulate the wave-like propagation dynamics critical to neural processing, particularly in the prefrontal cortex (PFC), where complex activity patterns emerge. BioOSS comprises two interacting populations of neurons: p neurons, which represent simplified membrane-potential-like units inspired by pyramidal cells in cortical columns, and o neurons, which govern propagation velocities and modulate the lateral spread of activity. Through local interactions, these neurons produce wave-like propagation patterns. The model incorporates trainable parameters for damping and propagation speed, enabling flexible adaptation to task-specific spatio-temporal structures. We evaluate BioOSS on both synthetic and real-world tasks, demonstrating superior performance and enhanced interpretability compared to alternative architectures.

Rhythm is an inherent aspect of human behavior, present from infancy and embedded in cultural practices. At the core of rhythm perception lies meter anticipation, a spontaneous process in the human brain that typically occurs before actual beats. This anticipation can be framed as a time series prediction problem. From the perspective of human embodied system behavior, although many models have been developed for time series prediction, most prioritize accuracy over biological realism, contrasting with the natural imprecision of human internal clocks. Neuroscientific evidence, such as infants' natural meter synchronization, underscores the need for biologically plausible models. Therefore, we propose a neuron oscillator-based dynamic system that simulates human behavior during meter perception. The model introduces two tunable parameters for local and global adjustments, fine-tuning the oscillation combinations to emulate human-like rhythmic behavior. The experiments are conducted under three common scenarios encountered during human-machine interaction, demonstrating that the proposed model can exhibit human-like reactions. Additionally, experiments involving human-machine and interhuman interactions show that the model successfully replicates real-world rhythmic behavior, advancing toward more natural and synchronized human-machine rhythm interaction.

This paper introduces a novel approach to predicting periodic time series using reservoir computing. The model is tailored to deliver precise forecasts of rhythms, a crucial aspect for tasks such as generating musical rhythm. Leveraging reservoir computing, our proposed method is ultimately oriented towards predicting human perception of rhythm. Our network accurately predicts rhythmic signals within the human frequency perception range. The model architecture incorporates primary and intermediate neurons tasked with capturing and transmitting rhythmic information. Two parameter matrices, denoted as c and k, regulate the reservoir’s overall dynamics. We propose a loss function to adapt c post-training and introduce a dynamic selection (DS) mechanism that adjusts k to focus on areas with outstanding contributions. Experimental results on a diverse test set showcase accurate predictions, further improved through real-time tuning of the reservoir via c and k. Comparative assessments highlight its superior performance compared to conventional models.

Learning to play a musical instrument such as a piano requires many hours of exercises, generally taken from a “method” book. These books are collections of progressive exercises intended to teach specific techniques and address the commonest mistakes and difficulties that players face while learning. One downside of these books is that the exercises are not personalized to the students and thus cannot address specific difficulties and characteristics of each learner. Given the many recent advances in the field of music generation, we propose that it should be possible to generate exercises automatically to form a personalized method for each student. The teacher would describe the characteristics of the student and their strengths and weaknesses to a software system, as well as the teaching goals that should be covered in the generated exercises, and the system would create exercises that are specific to the needs of the student and the concerns of the teacher, allowing for a more effective and engaging learning experience. In this paper, we describe a project trying to design such a system, stating research questions, describing the tentative methodology, and outlining its potential impact for both research in music generation and in computer-supported education.

Automatic Performer Identification from the symbolic representation of music has been a challenging topic in Music Information Retrieval (MIR).In this study, we apply a Recurrent Neural Network (RNN) model to classify the most likely music performers from their interpretative styles. We study different expressive parameters and investigate how to quantify these parameters for the exceptionally challenging task of performer identification. We encode performerstyle information using a Hierarchical Attention Network (HAN) architecture, based on the notion that traditional western music has a hierarchical structure (note, beat, measure, phrase level etc.). In addition, we present a large-scale dataset consisting of six virtuoso pianists performing the same set of compositions. The experimental results show that our model outperforms the baseline models with an F1-score of 0.845 and demonstrates the significance of the attention mechanism for understanding different performance styles.

New domain-specific languages for creating music and audio applications have typically been created in response to some technological challenge. Recent research has begun looking at how these languages impact our creative and aesthetic choices in music-making but we have little understanding on their effect on our wider programming practice. We present a survey that seeks to uncover what programming practices exist among interactive audio software developers and discover it is highly multi-practice, with developers adopting both exploratory programming and software engineering practice. A Q methodological study reveals that this multi-practice development is supported by different combinations of language features.

This article is about the relation between human observers and various human and non-human entities. Our focus is on humans' perception of movement. In particular how it affects the relationship to entities. We explore the way the movement of natural entities, locomoting animals and robots or the expressivity of dancers, play a vital part in our perception of these things. Humans' intuitive process of categorizing and attributing characteristics as a dialog and understanding of things, as found in the concept of metaphor, is central to our method. Drawing from the linguistic concept of animacy, expressing how sentient or alive an entity is interpreted we propose a metric of quantitative measures to investigate whether conceptual boundaries of entities, like those between human and non-human, change when movement comes into play. By means of measuring subjective responses, the rating of features in relation to specific types of entities like humans, animals and machines, we develop and validate a measurement tool in two online surveys. In the first (k = 93), we determine particular regions for each type, and in the second (k = 72), we investigate whether these regions change when entities move. We present the methodology and empirical work. Our key findings are alongside the metric, an agency-framework informed by related work to locate shifts in participants' interpretation as degrees of animacy and agency ranging from intentional action to causal movement. We provide results demonstrating the effect of participants' interpretation of entities under two conditions, represented either static or dynamic, we can show that movement affects participants' interpretation. For example the shift of a human represented with mechanical movement, by virtue of breakdancing moves, towards the region designated to machines.

The most commonly used formats for exchanging musical information today are limited in that they represent music as flat and rigid streams of events or as raw audio signals without any structural information about the content. Such files can only be listened to in a linear way and reused and manipulated in manners determined by a target application such as a Digital Audio Workstation. The publisher has no means to incorporate their intentions or understanding of the content. This article introduces an extension of the music formalism CHARM for the representation of modifiable and reusable musical content on the Web. It discusses how various kinds of multi-hierarchical graph structures together with logical constraints can be useful to model different musical situations. In particular, we focus on presenting solutions on how to interpret, navigate and schedule such structures in order for them to be played back. We evaluate the versatility of the representation in a number of practical examples created with a Web-based implementation based on Semantic Web technologies.

In this paper, a supervised approach based on Convolutional Neural Networks (CNN) for polyphonic piano transcription is presented. The system consists of pitch detection model, onset/offset detection model, and note search model. The pitch detection model is a single-channel CNN predicting the probabilities of pitches contained in one frame of the audio. The onset/offset model based on dual-channel CNN is used for estimating the probabilities of each pitch's onset or offset in a frame. The note search model is rule-based; it integrates the outputs of the pitch model and onset/offset model to determine the final onset, offset and pitch of notes in audio. Two experiments with different dataset conditions are accomplished to compare with state-of-the-art approaches on the same datasets. Experimental results reveal that the proposed approach preforms better in both frame- and note-based metrics.

An increasing number of tools are being developed to help academics interact with information, but little is known about the benefits of those tools for their users. This study evaluated academics' receptiveness to information proposed by a mobile app, the SerenA Notebook: information that is based in their inferred interests but does not relate directly to a prior recognized need. The evaluated app aimed at creating the experience of serendipitous encounters: generating ideas and inspiring thoughts, and potentially triggering follow‐up actions, by providing users with suggestions related to their work and leisure interests. We studied how 20 academics interacted with messages sent by the mobile app (3 per day over 10 consecutive days). Collected data sets were analyzed using thematic analysis. We found that contextual factors (location, activity, and focus) strongly influenced their responses to messages. Academics described some unsolicited information as interesting but irrelevant when they could not make immediate use of it. They highlighted filtering information as their major struggle rather than finding information. Some messages that were positively received acted as reminders of activities participants were meant to be doing but were postponing, or were relevant to ongoing activities at the time the information was received.

For a given piece of music, there often exist multiple versions belonging to the symbolic (e.g., MIDI representations), acoustic (audio recordings), or visual (sheet music) domain. Each type of information allows for applying specialized, domain-specific approaches to music analysis tasks. In this paper, we formulate the idea of a cross-version analysis for comparing and/or combining analysis results from different representations. As an example, we realize this idea in the context of harmonic analysis to automatically evaluate MIDI-based chord labeling procedures using annotations given for corresponding audio recordings. To this end, one needs reliable synchronization procedures that automatically establish the musical relationship between the multiple versions of a given piece. This becomes a hard problem when there are significant local deviations in these versions. We introduce a novel late-fusion approach that combines different alignment procedures in order to identify reliable parts in synchronization results. Then, the cross-version comparison of the various chord labeling results is performed only on the basis of the reliable parts. Finally, we show how inconsistencies in these results across the different versions allow for a quantitative and qualitative evaluation, which not only indicates limitations of the employed chord labeling strategies but also deepens the understanding of the underlying music material.

We argue that it is time to re-evaluate the MIR community's approach to building artificial systems which operate over music. We suggest that it is fundamentally problematic to view music simply as data representing audio signals, and that the notion of the so-called "semantic gap" is misleading. We propose a philosophical framework within which scientific and/or technological study of music can be carried out, free from such artificial constructions. Ultimately, we argue that Music (as opposed to sound) can be studied only in a context which explicitly allows for, and is built on, (albeit de facto) models of human perception; to do otherwise is not to study Music at all.