Abstract We present a platform and a dataset to help research on Music Emotion Recognition (MER). We developed the Music Enthusiasts platform aiming to improve the gathering and analysis of the so-called “ground truth” needed as input to MER systems. Firstly, our platform involves engaging participants using citizen science strategies and generate music emotion annotations – the platform presents didactic information and musical recommendations as incentivization, and collects data regarding demographics, mood, and language from each participant. Participants annotated each music excerpt with single free-text emotion words (in native language), distinct forced-choice emotion categories, preference, and familiarity. Additionally, participants stated the reasons for each annotation – including those distinctive of emotion perception and emotion induction. Secondly, our dataset was created for personalized MER and contains information from 181 participants, 4721 annotations, and 1161 music excerpts. To showcase the use of the dataset, we present a methodology for personalization of MER models based on active learning. The experiments show evidence that using the judgment of the crowd as prior knowledge for active learning allows for more effective personalization of MER systems for this particular dataset. Our dataset is publicly available and we invite researchers to use it for testing MER systems.

Our previous research showed promising results when transferring features learned from speech to train emotion recognition models for music. In this context, we implemented a denoising autoencoder as a pretraining approach to extract features from speech in two languages (English and Mandarin). From that, we performed transfer and multi-task learning to predict classes from the arousal-valence space of music emotion. We tested and analyzed intra-linguistic and cross-linguistic settings, depending on the language of speech and lyrics of the music. This paper presents additional investigation on our approach, which reveals that: (1) performing pretraining with speech in a mixture of languages yields similar results than for specific languages - the pretraining phase appears not to exploit particular language features, (2) the music in Mandarin dataset consistently results in poor classification performance - we found low agreement in annotations, and (3) novel methodologies for representation learning (Contrastive Predictive Coding) may exploit features from both languages (i.e., pretraining on a mixture of languages) and improve classification of music emotions in both languages. From this study we conclude that more research is still needed to understand what is actually being transferred in these type of contexts.

Selective hearing (SH) refers to the listeners' capability to focus their attention on a specific sound source or a group of sound sources in their auditory scene. This in turn implies that the listeners' focus is minimized for sources that are of no interest. This paper describes the current landscape of machine listening research, and outlines ways in which these technologies can be leveraged to achieve SH with computational means. To do so, a brief overview of the state-of-the-art in the fields of detection, classification, separation, localization and enhancement of sound sources is presented, highlighting recent advances in each field, and drawing connections between them. Two main challenges lie ahead in the development of SH applications: (1) Unified methods that can jointly detect/classify/localize and separate/enhance sound sources are required to provide both the flexibility and robustness required for real-life SH. (2) Low-latency methods suitable for real-time performance are critical when dealing with the dynamic nature of real-life auditory scenes.

Quality of sound source separation algorithms has traditionally been evaluated based on a set of established quality metrics: target distortion, interference from other sources, artifacts distortion, and a measure of overall separation quality. In our previous work, listening test results were presented where no significant correlation between these quality metrics and perceptual ratings from the listening test could be observed. Following these results, we now attempt to better understand perceptual quality in a sound source separation context. We focus on determining how separation quality is actually defined by listeners and propose the use of a descriptive methodology to reveal its most relevant dimensions. A combination of Free-Choice Profiling and Repertory Grid Technique is used with 10 human listeners in an attempt to verify if the main dimensions of separation quality truly correspond to those established by the quality metrics. The outcomes of this exploration bring light to the development of new methodologies for sound separation quality evaluation, and suggest a two-dimensional perceptual space for quality of sound source separation.

Both the collection and analysis of large music repertoires constitute major challenges within musicological disciplines such as jazz research. Automatic methods of music analysis based on audio signal processing have the potential to assist researchers and to accelerate the transcription and analysis of music recordings significantly. In this paper, we propose a framework for analyzing improvised monophonic solos in multi-instrumental jazz recordings with special focus on reed and brass instruments. The analysis algorithms rely on prior score-information, which is taken from high quality manual solo transcriptions. Following an initial solo and accompaniment source separation, we propose algorithms for tone-wise extraction of fundamental frequency and intensity contours. Based on this fine-grained representation of recorded jazz solos, we perform several exploratory experiments motivated by questions relating to jazz research in order to analyze the use of expressive stylistic devices such as intonation, pitch modulation, and dynamics in jazz solos. The results show that a score-informed audio analysis of jazz recordings can provide valuable insights into the individual stylistic characteristics of jazz musicians.

1628