Music recommendation has been a core area of interest within the recommender systems community since its early days. With the rise of music streaming platforms, algorithmic recommendations have become central to the music industry. However, critical challenges remain concerning the design, evaluation, and societal impact of music recommender systems. This third edition of the Music Recommender Systems Workshop (MuRS) centers on the growing influence of generative content on music recommendation. The rapid influx of AI-generated music transforms the streaming ecosystem, raising critical questions about discoverability, authenticity, and the curational role of recommender systems. The challenges and opportunities associated with AI-generated content extend beyond music and recommender systems, demanding transparent, fair, and accountable recommendation frameworks that reflect the interests of the diverse set of stakeholders.

As recommender systems are prone to various biases, mitigation approaches are needed to ensure that recommendations are fair to various stakeholders. One particular concern in music recommendation is artist gender fairness. Recent work has shown that the gender imbalance in the sector translates to the output of music recommender systems, creating a feedback loop that can reinforce gender biases over time.

Music recommendation has been relevant to the Recommender Systems (RecSys) community since the early days. With the growth of music streaming platforms, algorithmic recommendations have become critical in the music industry. However, many challenges are still wide open in the area of music recommender systems. Such challenges are currently being addressed in several research communities, including and beyond the RecSys and the Music Information Retrieval (MIR) communities. The RecSys conference has traditionally not focused very much on music content understanding. In contrast, while music content understanding is central to the MIR community, research on recommender systems is not prominent in MIR research. The Music Recommender Systems Workshop (MuRS) aims at bridging the existing gap between the diverse research communities focused on the specific challenges of music recommender systems. The workshop provides a space for researchers and practitioners from multiple disciplines to jointly discuss and exchange perspectives and solutions, and to promote discussion from both academia and industry upon future research directions in the area of music recommender systems.

Recommender systems have become the dominant means of curating cultural content, significantly influencing the nature of individual cultural experience. While the majority of academic and industrial research on recommender systems optimizes for personalized user experience, this paradigm does not capture the ways that recommender systems impact cultural experience in the aggregate, across populations of users. Although existing novelty, diversity, and fairness studies probe how recommender systems relate to the broader social role of cultural content, they do not adequately center culture as a core concept and challenge. In this work, we introduce commonality as a new measure of recommender systems that reflects the degree to which recommendations familiarize a given user population with specified categories of cultural content. Our proposed commonality metric responds to a set of arguments developed through an interdisciplinary dialogue between researchers in computer science and the social sciences and humanities. With reference to principles underpinning public service media (PSM) systems in democratic societies, we identify universality of address and content diversity in the service of strengthening cultural citizenship as particularly relevant goals for recommender systems delivering cultural content. We develop commonality as a measure of recommender system alignment with the promotion of a shared cultural experience of, and exposure to, diverse cultural content across a population of users. Moreover, we advocate for the involvement of human editors accountable to a larger value community as a fundamental part of defining categories in the service of cultural citizenship. We empirically compare the performance of recommendation algorithms using commonality with existing utility, diversity, novelty, and fairness metrics using three different domains. Our results demonstrate that commonality captures a property of system behavior complementary to existing metrics and suggests the need for alternative, non-personalized interventions in recommender systems oriented to strengthening cultural citizenship across populations of users. Moreover, commonality demonstrates both consistent results under different editorial policies and robustness to missing labels and users. Alongside existing fairness and diversity metrics, commonality contributes to a growing body of scholarship developing “public good” rationales for digital media and machine learning systems.

Evaluating recommender systems adequately and thoroughly is an important task. Significant efforts are dedicated to proposing metrics, methods, and protocols for doing so. However, there has been little discussion in the recommender systems’ literature on the topic of testing. In this work, we adopt and adapt concepts from the software testing domain, e.g., code coverage, metamorphic testing, or property-based testing, to help researchers to detect and correct faults in recommendation algorithms. We propose a test suite that can be used to validate the correctness of a recommendation algorithm, and thus identify and correct issues that can affect the performance and behavior of these algorithms. Our test suite contains both black box and white box tests at every level of abstraction, i.e., system, integration, and unit. To facilitate adoption, we release RecPack Tests , an open-source Python package containing template test implementations. We use it to test four popular Python packages for recommender systems: RecPack , PyLensKit , Surprise , and Cornac . Despite the high test coverage of each of these packages, we find that we are still able to uncover undocumented functional requirements and even some bugs. This validates our thesis that testing the correctness of recommendation algorithms can complement traditional methods for evaluating recommendation algorithms.

Music recommendation has been a prominent use case in the Rec-Sys community since the early days [4,14].With the growth of music streaming platforms in the last twenty years, algorithmic recommendation became critically important for the music industry.For consumers, when tenth of millions of music items are readily available, recommender systems are absolutely essential in helping to reduce the choice overload.Further, beyond assisting the listener in their music discovery, recommender systems have expanded to many aspects of the musical experience.A virtuous influential circle between the music industry and technological research drove improvements both in music listening experiences and in general scientific knowledge in recommender systems.Many fundamental topics in RecSys have matured together with their applications in music streaming (e.g.collaborative filtering, user modeling, etc.), while some distinctive aspects of the music medium (i.e.often consumed in sequence, passively, re-recommendation possible, etc. [13]) drove their own specific topics, such as playlist generation [5] or next-song recommendation [15].Today, music recommendation is a vibrant research area, prolific with respect to new topics [13] that led to novel contributions to the RecSys community.For example, in 2022 one of the best paper awards focused on understanding ways that recommendation

Recommender systems have become the dominant means of curating cultural content, significantly influencing the nature of individual cultural experience. While the majority of academic and industrial research on recommender systems optimizes for personalized user experience, this paradigm does not capture the ways that recommender systems impact cultural experience in the aggregate, across populations of users. Although existing novelty, diversity, and fairness studies probe how recommender systems relate to the broader social role of cultural content, they do not adequately center culture as a core concept and challenge. In this work, we introduce commonality as a new measure of recommender systems that reflects the degree to which recommendations familiarize a given user population with specified categories of cultural content. Our proposed commonality metric responds to a set of arguments developed through an interdisciplinary dialogue between researchers in computer science and the social sciences and humanities. With reference to principles underpinning non-profit, public service media (PSM) systems in democratic societies, we identify universality of address and content diversity in the service of strengthening cultural citizenship as particularly relevant goals for recommender systems delivering cultural content. Taking diversity in movie recommendation as a case study in enhancing pluralistic cultural experience, we empirically compare the performance of recommendation algorithms using commonality and existing utility, diversity, novelty, and fairness metrics. Our results demonstrate that commonality captures a property of system behavior complementary to existing metrics and suggest the need for alternative, non-personalized interventions in recommender systems oriented to strengthening cultural citizenship across populations of users. In this way, commonality contributes to a growing body of scholarship developing ‘public good’ rationales for digital media and machine learning systems.

Offline evaluation of information retrieval and recommendation has traditionally focused on distilling the quality of a ranking into a scalar metric such as average precision or normalized discounted cumulative gain. We can use this metric to compare the performance of multiple systems for the same request. Although evaluation metrics provide a convenient summary of system performance, they also collapse subtle differences across users into a single number and can carry assumptions about user behavior and utility not supported across retrieval scenarios. We propose recall-paired preference (RPP), a metric-free evaluation method based on directly computing a preference between ranked lists. RPP simulates multiple user subpopulations per query and compares systems across these pseudo-populations. Our results across multiple search and recommendation tasks demonstrate that RPP substantially improves discriminative power while correlating well with existing metrics and being equally robust to incomplete data.

Sequential recommendation problems have received increased research interest in recent years. In such scenarios, the task is to suggest items to users to consume next, given their past interaction history, e.g., the next movie to watch or the next item to place in the shopping cart. A number of machine learning models were proposed recently for the task of sequential recommendation, with the latest ones based on deep learning techniques, in particular on Transformers. Given the often surprisingly competitive performance of simpler nearest-neighbor methods for the related problem of session-based recommendation, we investigate the use of nearest-neighbor methods for sequential recommendation problems. Our analysis on four datasets shows that nearest-neighbor methods achieve comparable or better performance than the recent Transformer-based bert4rec method on two of them. However, the deep learning method outperforms the simple methods for the two larger datasets, confirming previous hypotheses that neural methods work best when more data is available. As a further result of our experiments, we found additional evidence that sampled metrics must be used with care, as they may not be predictive of an algorithm ranking that would be observed with the non-sampled, full evaluation.

As recommender systems play an important role in everyday life, there is an increasing pressure that such systems are fair. Besides serving diverse groups of users, recommenders need to represent and serve item providers fairly as well. In interviews with music artists, we identified that gender fairness is one of the artists' main concerns. They emphasized that female artists should be given more exposure in music recommendations. We analyze a widely-used collaborative filtering approach with two public datasets-enriched with gender information-to understand how this approach performs with respect to the artists' gender. To achieve gender balance, we propose a progressive re-ranking method that is based on the insights from the interviews. For the evaluation, we rely on a simulation of feedback loops and provide an in-depth analysis using state-of-the-art performance measures and metrics concerning gen-der fairness.

Session-based recommendation is a problem setting where the task of a recommender system is to make suitable item suggestions based only on a few observed user interactions in an ongoing session. The lack of long-term preference information about individual users in such settings usually results in a limited level of personalization, where a small set of popular items may be recommended to many users. This repeated exposure of such a subset of the items through the recommendations may in turn lead to a reinforcement effect over time, and to a system which is not able to help users discover new content anymore to the desirable extent.

Recent advances in deep learning have yielded new approaches for music recommendation in the long tail. The new approaches are based on data related to the music content (i.e. the audio signal) and context (i.e. other textual information), from which it automatically obtains a representation in a latent space that is used to generate the recommendations. The authors of these new approaches have shown improved accuracies, thus becoming the new state-of-the-art for music recommendation in the long tail.