DRVN is a regression testing tool that aims to diversify the test scenarios (road maps) to execute for testing and validating self-driving cars. DRVN harnesses the power of convolutional neural networks to identify possible failing roads in a set of generated examples before applying a greedy algorithm that selects and prioritizes the most diverse roads during regression testing. Initial testing discovered that DRVN performed well against random-based test selection.

Counterfactual explanations offer an intuitive and straightforward way to explain black-box models and offer algorithmic recourse to individuals. To address the need for plausible explanations, existing work has primarily relied on surrogate models to learn how the input data is distributed. This effectively reallocates the task of learning realistic explanations for the data from the model itself to the surrogate. Consequently, the generated explanations may seem plausible to humans but need not necessarily describe the behaviour of the black-box model faithfully. We formalise this notion of faithfulness through the introduction of a tailored evaluation metric and propose a novel algorithmic framework for generating Energy-Constrained Conformal Counterfactuals that are only as plausible as the model permits. Through extensive empirical studies, we demonstrate that ECCCo reconciles the need for faithfulness and plausibility. In particular, we show that for models with gradient access, it is possible to achieve state-of-the-art performance without the need for surrogate models. To do so, our framework relies solely on properties defining the black-box model itself by leveraging recent advances in energy-based modelling and conformal prediction. To our knowledge, this is the first venture in this direction for generating faithful counterfactual explanations. Thus, we anticipate that ECCCo can serve as a baseline for future research. We believe that our work opens avenues for researchers and practitioners seeking tools to better distinguish trustworthy from unreliable models.

As audio machine learning outcomes are deployed in societally impactful applications, it is important to have a sense of the quality and origins of the data used. Noticing that being explicit about this sense is not trivially rewarded in academic publishing in applied machine learning domains, and neither is included in typical applied machine learning curricula, we present a study into dataset usage connected to the top-5 cited papers at the International Conference on Acoustics, Speech, and Signal Processing (ICASSP). In this, we conduct thorough depthfirst analyses towards origins of used datasets, often leading to searches that had to go beyond what was reported in official papers, and ending into unclear or entangled origins. Especially in the current pull towards larger, and possibly generative AI models, awareness of the need for accountability on data provenance is increasing. With this, we call on the community to not only focus on engineering larger models, but create more room and reward for explicitizing the foundations on which such models should be built.

Developments in the field of Artificial Intelligence (AI), and particularly large language models (LLMs), have created a ’perfect storm’ for observing ’sparks’ of Artificial General Intelligence (AGI) that are spurious. Like simpler models, LLMs distill meaningful representations in their latent embeddings that have been shown to correlate with external variables. Nonetheless, the correlation of such representations has often been linked to human-like intelligence in the latter but not the former. We probe models of varying complexity including random projections, matrix decompositions, deep autoencoders and transformers: all of them successfully distill information that can be used to predict latent or external variables and yet none of them have previously been linked to AGI. We argue and empirically demonstrate that the finding of meaningful patterns in latent spaces of models cannot be seen as evidence in favor of AGI. Additionally, we review literature from the social sciences that shows that humans are prone to seek such patterns and anthropomorphize. We conclude that both the methodological setup and common public image of AI are ideal for the misinterpretation that correlations between model representations and some variables of interest are ’caused’ by the model’s understanding of underlying ’ground truth’ relationships. We, therefore, call for the academic community to exercise extra caution, and to be keenly aware of principles of academic integrity, in interpreting and communicating about AI research outcomes.

We explore the alignment of organizational representatives involved in hiring processes with five different, commonly proposed fairness notions. In a qualitative study with 17 organizational professionals, for each notion, we investigate their perception of understandability, fairness, potential to increase diversity, and practical applicability in the context of early candidate selection in hiring. In this, we do not explicitly frame our questions as questions of algorithmic fairness, but rather relate them to current human hiring practice. As our findings show, while many notions are well understood, fairness, potential to increase diversity and practical applicability are rated differently, illustrating the importance of understanding the application domain and its nuances, and calling for more interdisciplinary and human-centered research into the perception of mathematical fairness notions.

Adversarial examples remain a critical concern for the robustness of deep learning models, showcasing vulnerabilities to subtle input manipulations. While earlier research focused on generating such examples using white-box strategies, later research focused on gradient-based black-box strategies, as models' internals often are not accessible to external attackers. This paper extends our prior work by exploring a gradient-free search-based algorithm for adversarial example generation, with particular emphasis on differential evolution (DE). Building on top of the classic DE operators, we propose five variants of gradient-free algorithms: a single-objective approach (), two multi-objective variations ( and ), and two many-objective strategies ( and ). Our study on five canonical image classification models shows that whilst variant remains the fastest approach, consistently produces more minimal adversarial attacks (i.e., with fewer image perturbations). Moreover, we found that applying a post-process minimization to our adversarial images, would further reduce the number of changes and overall delta variation (image noise).

The extent to which popularity bias is propagated by media recommender systems is a current topic within the community, as is the uneven propagation among users with varying interests for niche items. Recent work focused on exactly this topic, with movies being the domain of interest. Later on, two different research teams reproduced the methodology in the domains of music and books, respectively. The results across the different domains diverge. In this paper, we reproduce the three studies and identify four aspects that are relevant in investigating the differences in results: data, algorithms, division of users in groups and evaluation strategy. We run a set of experiments in which we measure general popularity bias propagation and unfair treatment of certain users with various combinations of these aspects. We conclude that all aspects account to some degree for the divergence in results, and should be carefully considered in future studies. Further, we find that the divergence in findings can be in large part attributed to the choice of evaluation strategy.

Explainability and interpretability are two critical aspects of decision support systems. Despite their importance, it is only recently that researchers are starting to explore these aspects. This paper provides an introduction to explainability and interpretability in the context of apparent personality recognition. To the best of our knowledge, this is the first effort in this direction. We describe a challenge we organized on explainability in first impressions analysis from video. We analyze in detail the newly introduced data set, evaluation protocol, proposed solutions and summarize the results of the challenge. We investigate the issue of bias in detail. Finally, derived from our study, we outline research opportunities that we foresee will be relevant in this area in the near future.

Inspired by the success of deploying deep learning in the fields of Computer Vision and Natural Language Processing, this learning paradigm has also found its way into the field of Music Information Retrieval. In order to benefit from deep learning in an effective, but also efficient manner, deep transfer learning has become a common approach. In this approach, it is possible to reuse the output of a pre-trained neural network as the basis for a new learning task. The underlying hypothesis is that if the initial and new learning tasks show commonalities and are applied to the same type of input data (e.g., music audio), the generated deep representation of the data is also informative for the new task. Since, however, most of the networks used to generate deep representations are trained using a single initial learning source, their representation is unlikely to be informative for all possible future tasks. In this paper, we present the results of our investigation of what are the most important factors to generate deep representations for the data and learning tasks in the music domain. We conducted this investigation via an extensive empirical study that involves multiple learning sources, as well as multiple deep learning architectures with varying levels of information sharing between sources, in order to learn music representations. We then validate these representations considering multiple target datasets for evaluation. The results of our experiments yield several insights into how to approach the design of methods for learning widely deployable deep data representations in the music domain.

The field of recommender systems has a lot to gain from the field of psychology. Indeed, many psychology researchers have investigated relations between models that describe humans and consumption preferences. One example of this is personality, which has been shown to be a valid construct to describe people. As a consequence, personality-based recommenders have already proven to be a lead toward improving recommendations, by adapting them to their users' traits.

Prior research from the field of music psychology has suggested that there are factors common to music preference beyond individual genres. Specifically, research has shown that self-reported ratings of preference for individual musical genres can be reduced to 4 or 5 dimensions, which in turn have been shown to correlate to relevant psychological constructs, such as personality. However, the number of dimensions emerging from multiple studies has varied despite the care taken in conducting such research. Data-driven approaches offer opportunities to further this line of research with actual listening data, at a scale and scope surpassing that of traditional psychological studies. Although listening data can be considered more direct and comprehensive evidence of listening preference, transforming this data into meaningful measurements is non-trivial. In the current paper, we report on investigations seeking to find interpretable underlying dimensions of music taste, using implicit large-scale listening data. Offering a critical reflection on potential researchers' degrees of freedom, we adopt an explicit systematic approach, investigating the impact of varying different parameters, analysis, and normalization techniques. More precisely, we consider various ways to extract listening preference information from two large, openly available datasets of music listening behavior, making use of principal component analysis and variational autoencoders to extract potential underlying dimensions. Results and implications are discussed in light of prior psychological theory, and the potential of user listening data to further research on music preference.

In this paper, we focus on event detection over the timeline of a music track. Such technology is motivated by the need for innovative applications such as searching, nonlinear access, and recommendation. Event detection over the timeline requires time-code level labels in order to train machine learning models. We use timed comments from SoundCloud, a modern social music sharing platform, to obtain these labels. While in this way the need for tedious and time-consuming manual labeling can be reduced, the challenge is that timed comments are subject to additional temporal noise, as they occur in the temporal neighborhood of the actual events. We investigate the utility of such noisy timed comments as training labels through a case study, in which we investigate three types of events in electronic dance music (EDM): drop, build, and break. These socially significant events play a key role in an EDM track's unfolding and are popular in social media circles. These events are interesting for detection, and here we leverage the timed comments generated in the course of the online social activity around them. We propose a two-stage learning method that relies on noisy timed comments and, given a music track, marks the events on the timeline. In the experiments, we focus, in particular, on investigating to which extent noisy timed comments can replace manually acquired expert labels. The conclusions we draw during this study provide useful insights that motivate further research in the field of event detection.

In this workshop, we will interactively and jointly investigate with the community how intrinsic creative and artistic values, technology and society are critical to one another, and how our present-day urban and digital societies can demonstrably be enriched and advanced by effective connections between these all. The workshop will include interactive discussions, as well as a hands-on outdoors making session.

In this paper, we address the challenge of identifying music suitable to accompany typical daily activities. We first derive a list of common activities by analyzing social media data. Then, an automatic approach is proposed to find music for these activities. Our approach is inspired by our experimentally acquired findings (a) that genre and instrument information, i.e., as appearing in the textual metadata, are not sufficient to distinguish music appropriate for different types of activities, and (b) that existing content-based approaches in the music information retrieval community do not overcome this insufficiency. The main contributions of our work are (a) our analysis of the properties of activity-related music that inspire our use of novel high-level features, e.g., drop-like events, and (b) our approach's novel method of extracting and combining low-level features, and, in particular, the joint optimization of the time window for feature aggregation and the number of features to be used. The effectiveness of the approach method is demonstrated in a comprehensive experimental study including failure analysis.

This paper presents an exploratory study of the perceptions users have of diversity and ordering in playlist recommendations. There is a match between the diversification approach used in the system, and importance that users placed on the item properties. Surprisingly, participants had no expectations of the songs being in a particular order in a playlist. We discuss possible explanations for this finding, refining the research agenda to consider which ordering choices are perceptible to users, and influence user satisfaction.

This paper reports on three demonstrators developed to provide personalized, enhanced experiences of classical music, within the EU FP7 project "Performances as Highly Enriched aNd Interactive Concert eXperiences" (PHENICX): (i) an interface for accessing supplemental multimodal material about music items, (ii) a recommender system for visualizations of classical music, and (iii) a recommender system for tagging. The personalization in all three demos is achieved through modeling users' personality and musical sophistication. The links to the web interfaces are provided as well as the outcomes of quantitative and qualitative evaluations.

To enhance the experience of listening to classical orchestra music, either in the concert hall or at home, we present a personalized system that integrates three visualization/interaction concepts: Score Follower (points to the current position in the score), Orchestra Layout (illustrates instruments that are currently playing and their dynamics), and Structure Visualization (visualizes structural elements such as themes or motifs). Motivated by previous literature that found evidence for connections between personality and music consumption and preference, we first assessed in a user study to which extent personality traits and music visualization preferences correlate. Measuring preference via pragmatic quality and personality traits according to the Big Five Inventory (BFI) questionnaire, we found substantial interconnections between them. These translate into rules relating certain personality traits (e.g., extraversion or agreeableness) to preference rankings of the visualizations.

Information on whether a musician in a large symphonic orchestra plays her instrument at a given time stamp or not is valuable for a wide variety of applications aiming at mimicking and enriching the classical music concert experience on modern multimedia platforms. In this work, we propose a novel method for generating playing/non-playing labels per musician over time by efficiently and effectively combining an automatic analysis of the video recording of a symphonic concert and human annotation. In this way, we address the inherent deficiencies of traditional audio-only approaches in the case of large ensembles, as well as those of standard human action recognition methods based on visual models. The potential of our approach is demonstrated on two representative concert videos (about 7 hours of content) using a synchronized symbolic music score as ground truth. In order to identify the open challenges and the limitations of the proposed method, we carry out a detailed investigation of how different modules of the system affect the overall performance.

We present a system that we call ‘The Piano Music Companion’ and that is able to follow and understand (at least to some extent) a live piano performance. Within a few seconds this system can identify the piece that is being played, and the position within the piece. It then tracks the progress of the performer over time via a robust score following algorithm. The companion is useful in multiple ways, e.g., it can be used for piece identification, music visualisation, during piano rehearsal and for automatic page turning.

This paper presents the MusiClef data set, a multimodal data set of professionally annotated music. It includes editorial metadata about songs, albums, and artists, as well as MusicBrainz identifiers to facilitate linking to other data sets. In addition, several state-of-the-art audio features are provided. Different sets of annotations and music context data -- collaboratively generated user tags, web pages about artists and albums, and the annotation labels provided by music experts -- are included too. Versions of this data set were used in the MusiClef evaluation campaigns in 2011 and 2012 for auto-tagging tasks. We report on the motivation for the data set, on its composition, on related sets, and on the evaluation campaigns in which versions of the set were already used. These campaigns likewise represent one use case, i.e. music auto-tagging, of the data set. The complete data set is publicly available for download at http://www.cp.jku.at/musiclef.

In this extended abstract, we present a novel story-driven approach to soundtrack retrieval for user-generated videos. Cinematic knowledge on cross-modal associations is exploited through folksonomic story text retrieval from collaborative online metadata resources. Subsequently, audiovisual synchronization is applied based on high-level features described by users. The approach is demonstrated in the MuseSync prototype system.

The International ACM Workshop on Music Information Retrieval with User-Centered and Multimodal Strategies (MIRUM) at ACM Multimedia was proposed in order to gather experts from the Music and Multimedia Information Retrieval communities, as well as other neighboring fields, and to provide a high-profile platform for presenting current work on Music Information Retrieval with a strong focus on user-centered and multimodal approaches. Following a successful first edition at ACM Multimedia 2011, a second edition of MIRUM was held at ACM Multimedia 2012, which is the focus of this overview. After a description of the rationale and focus areas of the workshop, the accepted submissions and other program elements are summarized.

The 1st International ACM Workshop on Music Information Retrieval with User-Centered and Multimodal Strategies (MIRUM) at ACM Multimedia was proposed in order to gather experts from the Music and Multimedia Information Retrieval communities, as well as other neighboring fields. The workshop aims to provide a high-profile platform for presenting current work on Music Information Retrieval, with strong focus on user-centered and multimodal approaches. These focus areas are not only relevant to the Music Information Retrieval field, but equally recognized as emerging and relevant in the Multimedia domain. This way, a cross-disciplinary dialogue on open challenges can be initiated, facilitating new bridging opportunities and increased exchanges of expertise between communities. In this summary, we provide an overview of the 1st MIRUM workshop. After a description of the rationale and focus areas of the workshop, the accepted submissions and other program elements are summarized.