Diffusion recommender systems achieve strong recommendation accuracy but often suffer from popularity bias, resulting in unequal item exposure. To address this shortcoming, we introduce A2G-DiffRec, a diffusion recommender that incorporates adaptive autoguidance, where the main model is guided by a less-trained version of itself. Instead of using a fixed guidance weight, A2G-DiffRec learns to adaptively weigh the outputs of the main and weak models during training, supervised by a fairness-aware regularization that promotes balanced exposure across items with different popularity levels. Experimental results on three public datasets show that A2G-DiffRec is effective in enhancing item-side fairness at a marginal cost of accuracy reduction compared to existing guided diffusion recommenders and other non-diffusion baselines.

Music is characterized by aspects related to different modalities, such as the audio signal, the lyrics, or the music video clips. This has motivated the development of multimodal datasets and methods for Music Information Retrieval (MIR) tasks such as genre classification or autotagging. Music can be described at different levels of granularity, for instance defining genres at the level of artists or music albums. However, most datasets for multimodal MIR neglect this aspect and provide data at the level of individual music tracks. We aim to fill this gap by providing Music4All Artist and Album (Music4All A+A), a dataset for multimodal MIR tasks based on music artists and albums. Music4All A+A is built on top of the Music4All-Onion dataset, an existing track-level dataset for MIR tasks. Music4All A+A provides metadata, genre labels, image repre-sentations, and textual descriptors for 6,741 artists and 19,511 albums. Furthermore, since Music4All A+A is built on top of Music4AII-Onion, it allows access to other multimodal data at the track level, including user-item interaction data. This renders Music4All A+A suitable for a broad range of MIR tasks, including multimodal music recommendation, at several levels of granularity. To showcase the use of Music4All A+A, we carry out experiments on multimodal genre classification of artists and albums, including an analysis in missing-modality scenarios, and a quantitative comparison with genre classification in the movie domain. Our experiments show that images are more informative for classifying the genres of artists and albums, and that several multimodal models for genre classification struggle in generalizing across domains. We provide the code to reproduce our experiments at https://github.com/hcai-mmslMusic4AlI-A-A, the dataset will be linked in the repository at camera-ready and provided open-source under a CC BY-NC-SA 4.0 license.

Explanatory interactive learning (XIL) enables users to guide model training in machine learning (ML) by providing feedback on the model's explanations, thereby helping it to focus on features that are relevant to the prediction from the user's perspective. In this study, we explore the capability of this learning paradigm to mitigate bias and spurious correlations in visual classifiers, specifically in scenarios prone to data bias, such as gender classification. We investigate two methodologically different state-of-the-art XIL strategies, i.e., CAIPI and Right for the Right Reasons (RRR), as well as a novel hybrid approach that combines both strategies. The results are evaluated quan-titatively by comparing segmentation masks with explanations generated using Gradient-weighted Class Activation Mapping (GradCAM) and Bounded Logit Attention (BLA). Experimental results demonstrate the effectiveness of these methods in (i) guiding ML models to focus on relevant image features, par-ticularly when CAIPI is used, and (ii) reducing model bias (i.e., balancing the misclassification rates between male and female predictions). Our analysis further supports the potential of XIL methods to improve fairness in gender classifiers. Overall, the increased transparency and fairness obtained by XIL leads to slight performance decreases with an exception being CAIPI, which shows potential to even improve classification accuracy.

Music psychology has identified various reasons why people listen to music, based on empirical evidence from interviews and surveys.In this paper, we take a data-driven approach that adopts both pre-trained Sentence Transformers and Cross Encoder, as well as graph-based clustering to first determine music listening intents and then explore user-generated playlists by comparing the title to the listening intents.For this purpose, we first investigated whether 129 established listening functions, previously identified by Schäfer et al. (2013), could be meaningfully clustered into broader listening intents.While Schäfer et al. (2013) introduced three broad dimensions of music listening, this work aimed to identify smaller, context-specific intents to capture more nuanced intents.The resulting clusters were then evaluated through a first survey to select the clusters of the best performing model.In a second survey, music listening intent clusters were explored in more detail to obtain a deeper understanding of their significance for music retrieval and recommendation.Lastly, the playlist selection per intent and characteristics of different listening intents were further explored through a third survey.Given the encouraging results of the evaluation of the computed clusters (92% of clusters judged consistent by participants) and the insight that more than half of the participants search for playlists for a specific intent, we propose a browsing system that categorizes playlists based on their intent and enables users to explore similar playlists.Our approach is further visualized in a dashboard to explore and browse through playlists in intent space.

Calibration is the degree to which a recommender system is able to match the distribution of a certain item attribute among the items consumed by a user with their respective recommendations.Recent work suggests that many recommenders tend to provide miscalibrated recommendations.Furthermore, most approaches aimed at improving calibration adopt the post-processing paradigm, making them computationally costly at the inference time.This work proposes CaliTune, a fine-tuning approach applied to collaborative filtering based recommenders to allow them generate better calibrated recommendations without relying on costly post-processing.We compare CaliTune to an established post-processing approach on two backbone models and datasets from movie and music domains, focusing on popularity calibration.Our results suggest that CaliTune can offer a competitive accuracy-calibration trade-off in several settings, particularly when the backbone model exhibits high miscalibration and accuracy remains important, making it a promising inference-efficient alternative in such cases.

Natural language interfaces offer a compelling approach for music recommendation, enabling users to express complex preferences conversationally. While Large Language Models (LLMs) show promise in this direction, their scalability in recommender systems is limited by high costs and latency. Retrieval-based approaches using smaller language models mitigate these issues but often rely on single-modal item representations, overlook long-term user preferences, and require full model retraining, posing challenges for real-world deployment. In this paper, we present JAM (Just Ask for Music), a lightweight and intuitive framework for natural language music recommendation. JAM models user-query-item interactions as vector translations in a shared latent space, inspired by knowledge graph embedding methods like TransE. To capture the complexity of music and user intent, JAM aggregates multimodal item features via cross-attention and sparse mixture-of-experts. We also introduce JAMSessions, a new dataset of over 100k user-query-item triples with anonymized user/item embeddings, uniquely combining conversational queries and user long-term preferences. Our results show that JAM provides accurate recommendations, produces intuitive representations suitable for practical use cases, and can be easily integrated with existing music recommendation stacks.

Recommender Systems (RS) often rely on representations of users and items in a joint embedding space and on a similarity metric to compute relevance scores.In modern RS, the modules to obtain user and item representations consist of two distinct and separate neural networks (NN).In multimodal representation learning, weight sharing has been proven effective in reducing the distance between multiple modalities of a same item.Inspired by these approaches, we propose a novel RS that leverages weight sharing between the user and item NN modules used to obtain the latent representations in the shared embedding space.The proposed framework consists of a single Collaborative Branch for Recommendation (CoBraR).We evaluate CoBraR by means of quantitative experiments on ecommerce and movie recommendation.Our experiments show that by reducing the number of parameters and improving beyondaccuracy aspects without compromising accuracy, CoBraR has the potential to be applied and extended for real-world scenarios.

Psychological processes play a critical role in shaping users' interactions with information retrieval (IR) and recommender systems (RS). Therefore, understanding human cognition, decision-making, and emotions is vital to enable user-centric retrieval and recommendation systems. Vice versa, understanding whether these aspects are also present in the systems themselves (e.g., in training data, ranking models, or outputs), or even injecting them on purpose, can inform the development of psychology-inspired systems. The purpose of this tutorial is to provide its attendees with an introduction to psychological concepts that are important in the ecosystem of search, retrieval, and recommendation, in particular, cognitive architectures, cognitive effects and biases, as well as personality and affect. Leveraging corresponding models allows its audience to build or refine psychology-informed IR and RS technology. The interdisciplinary tutorial requires intermediate expertise in terms of IR and RS, while we do not assume knowledge in psychology.

Music consumption is shaped by both internal factors (e.g., mood, motivation) and external factors (e.g., activity, social environment), which together influence listeners' behavior (e.g., focus, songs' skips) and reactions (e.g., mood changes).While prior research has explored real-life or survey-based, context-aware music listening with limited available context information, we introduce a dataset comprising 216 music listening sessions collected in real-world settings through a custom-built Android mobile application designed to assess a wide range of contextual factors.The dataset captures static (e.g., activity, social environment, motivation) and dynamic (e.g., mood changes) contextual factors, along with music interaction data (e.g., skipped or fully listened songs), listening focus levels, and participant traits (e.g., demographics, music education, listening preferences, personality).Our analysis highlights key insights into how different contextual factors influence user behavior and mood.demonstrating significant differences in skipping songs, focus levels, and genre diversity.We show that music listening sessions grouped by context are significantly different in terms of music listening behaviors (focus, skipping, and session genre diversity) and mood changes (happiness, sadness, stress, and energy).Furthermore, we explore the correlations between personality traits and listening behaviors (mean skip rate and genre diversity).Ultimately, our findings emphasize the importance of understanding context, as different situations lead to distinct music preferences and have varying impacts on user behavior and emotional responses.

Humans have the tendency to discover and explore.This natural tendency is reflected in data from streaming platforms as the amount of previously unknown content accessed by users.Additionally, in domains such as that of music streaming there is evidence that recommending novel content improves users' experience with the platform.Therefore, understanding users' discovery patterns, such as the amount to which and the way users access previously unknown content, is a topic of relevance for both the scientific community and the streaming industry, particularly the music one.Previous works studied how music consumption differs for users of different traits and looked at diversity, novelty, and consistency over time of users' music preferences.However, very little is known about how users discover and explore previously unknown music, and how this behavior differs for users of varying discovery needs.In this paper we bridge this gap by analyzing data from a survey answered by users of the major music streaming platform Deezer in combination with their streaming data.We first address questions regarding whether users who declare a higher interest in unfamiliar music listen to more diverse music, have more stable music preferences over time, and explore more music within a same time window, compared to those who declare a lower interest.We then investigate which type of music tracks users choose to listen to when they explore unfamiliar music, identifying clear patterns of popularity and genre representativeness that vary for users of different discovery needs.Our findings open up possibilities to infer users' interest in unfamiliar music from streaming data as well as possibilities to develop recommender systems that guide users in exploring music in a more natural way .

This editorial introduces the Special Issue on Trustworthy Recommender Systems , hosted by the ACM Transactions on Recommender Systems in 2024. We provide an overview on the multifaceted aspects of trustworthiness and point to recent regulations that underline the importance of the topic, also beyond technical perspectives. Subsequently, we present the nine articles constituting the special issue: one survey that reviews over 400 papers, categorizing them according to five trustworthiness dimensions, and eight research articles . We categorize and introduce the latter according to the major trustworthiness dimensions they address, specifically into privacy/security , transparency/explainability , and bias/fairness . We provide a summary of their main contributions and end with a brief personal statement about envisioned challenges ahead.

In today's data-rich environment, visualization literacy-the ability to understand and communicate information through charts-is increasingly important. However, constructing effective charts can be challenging due to the numerous design choices involved. Off-the-shelf systems and libraries produce charts with carefully selected defaults that users may not be aware of, making it hard to increase their visualization literacy with those systems. In addition, traditional ways of improving visualization literacy, such as textbooks and tutorials, can be burdensome as they require sifting through a plethora of resources. To address this challenge, we designed Iguanodon, an easy-to-use game application that complements the traditional methods of improving visualization construction literacy. In our game application, users interactively choose whether to apply design choices, which we assign to sub-tasks that must be optimized to create an effective chart. The application offers multiple game variations to help users learn how different design choices should be applied to construct effective charts. Furthermore, our approach easily adapts to different visualization design guidelines. We describe the application's design and present the results of a user study with 37 participants. Our findings indicate that our game-based approach supports users in improving their visualization literacy.

Shahed Masoudian, Cornelia Volaucnik, Markus Schedl, Navid Rekabsaz. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Collaborative filtering-based recommender systems leverage vast amounts of behavioral user data, which poses severe privacy risks. Thus, often random noise is added to the data to ensure Differential Privacy (DP). However, to date, it is not well understood in which ways this impacts personalized recommendations. In this work, we study how DP affects recommendation accuracy and popularity bias when applied to the training data of state-of-the-art recommendation models. Our findings are three-fold: First, we observe that nearly all users’ recommendations change when DP is applied. Second, recommendation accuracy drops substantially while recommended item popularity experiences a sharp increase, suggesting that popularity bias worsens. Finally, we find that DP exacerbates popularity bias more severely for users who prefer unpopular items than for users who prefer popular items.

Many IR systems project harmful societal biases, including gender bias, in their retrieved contents. Uncovering and addressing such biases requires grounded bias measurement principles. However, defining reliable bias metrics for search results is challenging, particularly due to the difficulties in capturing gender-related tendencies in the retrieved documents. In this work, we propose a new framework for search result bias measurement. Within this framework, we first revisit the current metrics for representative search result bias (RepSRB) that are based on the occurrence of gender-specific language in the search results. Addressing their limitations, we additionally propose a metric for comparative search result bias (ComSRB) measurement and integrate it into our framework. ComSRB defines bias as the skew in the set of retrieved documents in response to a non-gendered query toward those for male/female-specific variations of the same query. We evaluate ComSRB against RepSRB on a recent collection of bias-sensitive topics and documents from the MS MARCO collection, using pre-trained bi-encoder and cross-encoder IR models. Our analyses show that, while existing metrics are highly sensitive to the wordings and linguistic formulations, the proposed ComSRB metric mitigates this issue by focusing on the deviations of a retrieval list from its explicitly biased variants, avoiding the need for sub-optimal content analysis processes.

Segmenting text into sentences plays an early and crucial role in many NLP systems.This is commonly achieved by using rule-based or statistical methods relying on lexical features such as punctuation.Although some recent works no longer exclusively rely on punctuation, we find that no prior method achieves all of (i) robustness to missing punctuation, (ii) effective adaptability to new domains, and (iii) high efficiency.We introduce a new model -Segment any Text (SAT) -to solve this problem.To enhance robustness, we propose a new pretraining scheme that ensures less reliance on punctuation.To address adaptability, we introduce an extra stage of parameter-efficient fine-tuning, establishing state-of-the-art performance in distinct domains such as verses from lyrics and legal documents.Along the way, we introduce architectural modifications that result in a threefold gain in speed over the previous state of the art and solve spurious reliance on context far in the future.Finally, we introduce a variant of our model with fine-tuning on a diverse, multilingual mixture of sentence-segmented data, acting as a drop-in replacement and enhancement for existing segmentation tools.Overall, our contributions provide a universal approach for segmenting any text.Our method outperforms all baselines -including strong LLMs -across 8 corpora spanning diverse domains and languages, especially in practically relevant situations where text is poorly formatted.1

Language models frequently inherit societal biases from their training data.Numerous techniques have been proposed to mitigate these biases during both the pre-training and finetuning stages.However, fine-tuning a pretrained debiased language model on a downstream task can reintroduce biases into the model.Additionally, existing debiasing methods for downstream tasks either (i) require labels of protected attributes (e.g., age, race, or political views) that are often not available or (ii) rely on indicators of bias, which restricts their applicability to gender debiasing since they rely on gender-specific words.To address this, we introduce a novel debiasing regularization technique based on the class-wise variance of embeddings.Crucially, our method does not require attribute labels and targets any attribute, thus addressing the shortcomings of existing debiasing methods.Our experiments on encoder language models and three datasets demonstrate that our method outperforms existing strong debiasing baselines that rely on target attribute labels while maintaining performance on the target task.1

Most recommender systems adopt collaborative filtering (CF) and provide recommendations based on past collective interactions. Therefore, the performance of CF algorithms degrades when few or no interactions are available, a scenario referred to as cold-start. To address this issue, previous work relies on models leveraging both collaborative data and side information on the users or items. Similar to multimodal learning, these models aim at combining collaborative and content representations in a shared embedding space. In this work we propose a novel technique for multimodal recommendation, relying on a multimodal Single-Branch embedding network for Recommendation (SiBraR). Leveraging weight-sharing, SiBraR encodes interaction data as well as multimodal side information using the same single-branch embedding network on different modalities. This makes SiBraR effective in scenarios of missing modality, including cold start. Our extensive experiments on large-scale recommendation datasets from three different recommendation domains (music, movie, and e-commerce) and providing multimodal content information (audio, text, image, labels, and interactions) show that SiBraR significantly outperforms CF as well as state-of-the-art content-based RSs in cold-start scenarios, and is competitive in warm scenarios. We show that SiBraR’s recommendations are accurate in missing modality scenarios, and that the model is able to map different modalities to the same region of the shared embedding space, hence reducing the modality gap.

Recent work suggests that music recommender systems are prone to disproportionally frequent recommendations of music from countries more prominently represented in the training data, notably the US. However, it remains unclear to what extent feedback loops in music recommendation influence the dynamics of such imbalance. In this work, we investigate the dynamics of representation of local (i.e., country-specific) and US-produced music in user profiles and recommendations. To this end, we conduct a feedback loop simulation study using the LFM-2b dataset. The results suggest that most of the investigated recommendation models decrease the proportion of music from local artists in their recommendations. Furthermore, we find that models preserving average proportions of US and local music do not necessarily provide country-calibrated recommendations. We also look into popularity calibration and, surprisingly, find that the most popularity-calibrated model in our study (ItemKNN) provides the least country-calibrated recommendations. In addition, users from less represented countries (e.g., Finland) are, in the long term, most affected by the under-representation of their local music in recommendations.

Emotions constitute an important aspect when listening to music. While manual annotations from user studies grounded in psychological research on music and emotions provide a well-defined and fine-grained description of the emotions evoked when listening to a music track, user-generated tags provide an alternative view stemming from large-scale data. In this work, we examine the relationship between these two emotional characterizations of music and analyze their impact on the performance of emotion-based music recommender systems individually and jointly. Our analysis shows that (i) the agreement between the two characterizations, as measured with Cohen’s κ coefficient and Kendall rank correlation, is often low, (ii) Leveraging the emotion profile based on the intensity of evoked emotions from high-quality annotations leads to performances that are stable across different recommendation algorithms; (iii) Simultaneously leveraging the emotion profiles based on high-quality and large-scale annotations allows to provide recommendations that are less exposed to the low accuracy that algorithms might reach when leveraging one type of data, only.

The Psychology-informed Information Access Systems (PsyIAS) workshop bridges the fields of machine learning and psychology, aiming to connect the research communities of information retrieval, recommender systems, natural language processing, as well as cognitive and behavioral psychology. It serves as a forum for multidisciplinary discussions about the use of psychological constructs, theories, and empirical findings for modeling and predicting user preferences, intents, and behaviors. PsyIAS particularly focuses on research that incorporates such psychology-inspired models into the search, retrieval, and recommendation processes, creates corresponding algorithms and systems, or looks into the role of cognitive processes underlying human information access. More information can be found at https://sites.google.com/view/psyias.

Biases in algorithmic systems have led to discrimination against historically disadvantaged groups, including the reinforcement of outdated gender stereotypes. While a substantial body of research addresses biases in algorithms and underlying data, in this work, we study if and how users themselves reflect these biases in their interactions with systems, which expectedly leads to the further manifestation of biases. More specifically, we investigate the replication of stereotypical gender representations by users in formulating online search queries. Following prototype theory, we define the disproportionate mention of the gender that does not conform to the prototypical representative of a searched domain (e.g., “male nurse”) as an indication of bias. In a pilot study with 224 US participants and a main study with 400 UK participants, we find clear evidence of gender biases in formulating search queries. We also report the effects of an educative text on user behaviour and highlight the wish of users to learn about bias-mitigating strategies in their interactions with search engines.

Recommender systems are designed to help us navigate through an abundance of online content. Collaborative filtering (CF) approaches are commonly used to leverage behaviors of others with a similar taste to make predictions for the target user. However, CF is prone to introduce or amplify popularity bias in which popular (often consumed or highly ranked) items are prioritized over less popular items. Many computational metrics of popularity biases — and resulting algorithmic (un)fairness — have been presented. However, it is largely unclear whether these metrics reflect human perception of bias and fairness. We conducted a user study with 170 participants to explore how users perceive recommendation lists created by algorithms with different degrees of popularity bias. Our results show — surprisingly — that popularity biases in recommendation lists are barely observed by users, even when corresponding bias/fairness metrics clearly indicate them.

The provided contents by information retrieval (IR) systems can reflect the existing societal biases and stereotypes. Such biases in retrieval results can lead to further establishing and strengthening stereotypes in society and also in the systems. To facilitate the studies of gender bias in the retrieval results of IR systems, we introduce Gender Representation-Bias for Information Retrieval (Grep-BiasIR), a novel thoroughly-audited dataset consisting of 118 bias-sensitive neutral search queries. The set of queries covers a wide range of gender-related topics, for which a biased representation of genders in the search result can be considered as socially problematic. Each query is accompanied with one relevant and one non-relevant document, where the document is also provided in three variations of female, male, and neutral. The dataset is available at https://github.com/KlaraKrieg/GrepBiasIR.

Deepak Kumar, Oleg Lesota, George Zerveas, Daniel Cohen, Carsten Eickhoff, Markus Schedl, Navid Rekabsaz. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. 2023.

Music listening sessions often consist of sequences including repeating tracks. Modeling such relistening behavior with models of human memory has been proven effective in predicting the next track of a session. However, these models intrinsically lack the capability of recommending novel tracks that the target user has not listened to in the past. Collaborative filtering strategies, on the contrary, provide novel recommendations by leveraging past collective behaviors but are often limited in their ability to provide explanations. To narrow this gap, we propose four hybrid algorithms that integrate collaborative filtering with the cognitive architecture ACT-R. We compare their performance in terms of accuracy, novelty, diversity, and popularity bias, to baselines of different types, including pure ACT-R, kNN-based, and neural-networks-based approaches. We show that the proposed algorithms are able to achieve the best performances in terms of novelty and diversity, and simultaneously achieve a higher accuracy of recommendation with respect to pure ACT-R models. Furthermore, we illustrate how the proposed models can provide explainable recommendations.

This tutorial provides an interdisciplinary overview about the topics of fairness, non-discrimination, transparency, privacy, and security in the context of recommender systems. These are important dimensions of trustworthy AI systems according to European policies, but also extend to the global debate on regulating AI technology. Since we strongly believe that the aforementioned aspects require more than merely technical considerations, we discuss these topics also from ethical, legal, and regulatory points of views, intertwining different perspectives. The main focus of the tutorial is still on presenting technical solutions that aim at addressing the mentioned topics of trustworthiness. In addition, the tutorial equips the mostly technical audience of RecSys with the necessary understanding of the social and ethical implications of their research and development, and of recent ethical guidelines and regulatory frameworks.

Popularity bias in recommendation lists refers to over-representation of popular content and is a challenge for many recommendation algorithms. Previous research has suggested several offline metrics to quantify popularity bias, which commonly relate the popularity of items in users' recommendation lists to the popularity of items in their interaction history. Discrepancies between these two factors are referred to as popularity miscalibration. While popularity metrics provide a straightforward and well-defined means to measure popularity bias, it is unknown whether they actually reflect users' perception of popularity bias.

We present Travel Bird, a novel personalized destination recommendation and exploration interface which allows its users to find their next tourist destination by describing their specific preferences in a narrative form. Unlike other solutions, Travel Bird is based on TourBERT, a novel NLP model we developed, specifically tailored to the tourism domain. Travel Bird creates a two-dimensional personalized destination exploration space from TourBERT embeddings of social media content and the users' textual description of the experience they are looking for. In this demo, we will showcase several use cases for Travel Bird, which are beneficial for consumers and destination management organizations.

This tutorial aims at providing its audience an interdisciplinary overview about the topics of fairness and non-discrimination, diversity, and transparency as relevant dimensions of trustworthy AI systems, tailored to algorithmic ranking systems such as search engines and recommender systems. We will equip the mostly technical audience of WSDM with the necessary understanding of the social and ethical implications of their research and development on the one hand, and of recent ethical guidelines and regulatory frameworks addressing the aforementioned dimensions on the other hand. While the tutorial foremost takes a European perspective, starting from the concept of trustworthy AI and discussing EU regulation in this area currently in the implementation stages, we also consider related initiatives worldwide. Since ensuring non-discrimination, diversity, and transparency in retrieval and recommendation systems is an endeavor in which academic institutions and companies in different parts of the world should collaborate, this tutorial is relevant for researchers and practitioners interested in the ethical, social, and legal impact of their work. The tutorial, therefore, targets both academic scholars and practitioners around the globe, by reviewing recent research and providing practical examples addressing these particular trustworthiness aspects, and showcasing how new regulations affect the audience's daily work.

User-based KNN recommender systems ( UserKNN ) utilize the rating data of a target user’s k nearest neighbors in the recommendation process. This, however, increases the privacy risk of the neighbors, since the recommendations could expose the neighbors’ rating data to other users or malicious parties. To reduce this risk, existing work applies differential privacy by adding randomness to the neighbors’ ratings, which unfortunately reduces the accuracy of UserKNN . In this work, we introduce ReuseKNN , a novel differentially private KNN-based recommender system. The main idea is to identify small but highly reusable neighborhoods so that (i) only a minimal set of users requires protection with differential privacy and (ii) most users do not need to be protected with differential privacy since they are only rarely exploited as neighbors. In our experiments on five diverse datasets, we make two key observations. Firstly, ReuseKNN requires significantly smaller neighborhoods and, thus, fewer neighbors need to be protected with differential privacy compared with traditional UserKNN . Secondly, despite the small neighborhoods, ReuseKNN outperforms UserKNN and a fully differentially private approach in terms of accuracy. Overall, ReuseKNN leads to significantly less privacy risk for users than in the case of UserKNN .

Personalized recommender systems are essential tools to facilitate human decision making. Many contemporary recommender systems use advanced machine learning techniques to model and predict user preferences from behavioral data. While such systems can provide helpful recommendations, their algorithms’ design does not incorporate the underlying psychological mechanisms that shape user preferences and behavior. In this tutorial, we will guide the attendees through the state-of-the-art in psychology-informed recommender systems, i.e., recommender systems that consider extrinsic and intrinsic human factors. We show how such systems can improve the recommendation process in a user-centric fashion.

We present the LFM-2b dataset containing the listening records of over 120,000 users of the music platform Last.fm. These users provide a total of more than two billion individual listening events that span a time range of over 15 years, from February 2005 until March 2020. These listening events refer to a total of 50 million distinct tracks of 5 million distinct artists. Beside the common metadata (i. e., artist and track name), LFM-2b contains additional information both regarding the users and items. This includes the demographic information of users, namely country, gender, and age, and the fine-grained genre and style of items together with the vector embeddings of their lyrics.

When we appreciate a piece of music, it is most naturally because of its content, including rhythmic, tonal, and timbral elements as well as its lyrics and semantics. This suggests that the human affinity for music is inherently content-driven. This kind of information is, however, still frequently neglected by mainstream recommendation models based on collaborative filtering that rely solely on user-item interactions to recommend items to users. A major reason for this neglect is the lack of standardized datasets that provide both collaborative and content information. The work at hand addresses this shortcoming by introducing Music4All-Onion, a large-scale, multi-modal music dataset. The dataset expands the Music4All dataset by including 26 additional audio, video, and metadata characteristics for 109,269 music pieces. In addition, it provides a set of 252,984,396 listening records of 119,140 users, extracted from the online music platform Last.fm, which allows leveraging user-item interactions as well. We organize distinct item content features in an onion model according to their semantics, and perform a comprehensive examination of the impact of different layers of this model (e.g., audio features, user-generated content, and derivative content) on content-driven music recommendation, demonstrating how various content features influence accuracy, novelty, and fairness of music recommendation systems. In summary, with Music4All-Onion, we seek to bridge the gap between collaborative filtering music recommender systems and content-centric music recommendation requirements.

We introduce Emotion-aware Music Tower Blocks (EmoMTB), an audiovisual interface to explore large music collections. It creates a musical landscape, by adopting the metaphor of a city, where similar songs are grouped into the same building and nearby buildings form neighborhoods of particular genres. In order to personalize the user experience, an underlying classifier monitors textual user-generated content, by predicting their emotional state and adapting the audiovisual elements of the interface accordingly. EmoMTB enables users to explore different musical styles either within their comfort zone or outside of it. Besides, tailoring the results of the recommender engine to match the affective state of the user, EmoMTB offers a unique way to discover and enjoy music. EmoMTB supports exploring a collection of circa half a million streamed songs using a regular smartphone as a control interface to navigate in the landscape.

Recommender systems are essential tools to support human decision-making in online information spaces. Many state-of-the-art recommender systems adopt advanced machine learning techniques to model and predict user preferences from behavioral data. While such systems can provide useful and effective recommendations, their algorithmic design commonly neglects underlying psychological mechanisms that shape user preferences and behavior. In this tutorial, we offer a comprehensive review of the state of the art and progress in psychology-informed recommender systems, i.e., recommender systems that incorporate human cognitive processes, personality, and affective cues into recommendation models, along with definitions, strengths and weaknesses. We show how such systems can improve the recommendation process in a user-centric fashion. With this tutorial, we aim to stimulate more ideas and discussion with the audience on core issues of this topic such as the identification of suitable psychological models, availability of datasets, or the suitability of existing performance metrics to evaluate the efficacy of psychology-informed recommender systems. Besides, we present takeaways to recommender systems practitioners how to build psychology-informed recommender systems. Previous versions of this tutorial were presented, among others, at The ACM Web Conference 2022 and the ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR) 2022.

Recent studies show the benefits of reformulating common machine learning models through the concept of prototypes – representatives of the underlying data, used to calculate the prediction score as a linear combination of similarities of a data point to prototypes. Such prototype-based formulation of a model, in addition to preserving (sometimes enhancing) the performance, enables explainability of the model’s decisions, as the prediction can be linearly broken down into the contributions of distinct definable prototypes. Following this direction, we extend the idea of prototypes to the recommender system domain by introducing ProtoMF, a novel collaborative filtering algorithm. ProtoMF learns sets of user/item prototypes that represent the general consumption characteristics of users/items in the underlying dataset. Using these prototypes, ProtoMF then represents users and items as vectors of similarities to the corresponding prototypes. These user/item representations are ultimately leveraged to make recommendations that are both effective in terms of accuracy metrics, and explainable through the interpretation of prototypes’ contributions to the affinity scores. We conduct experiments on three datasets to assess both the effectiveness and the explainability of ProtoMF. Addressing the former, we show that ProtoMF exhibits higher Hit Ratio and NDCG compared to other relevant collaborative filtering approaches. As for the latter, we qualitatively show how ProtoMF can provide explainable recommendations and how its explanation capabilities can expose the existence of statistical biases in the learned representations, which we exemplify for the case of gender bias.

Collaborative filtering algorithms capture underlying consumption patterns, including the ones specific to particular demographics or protected information of users, e.g., gender, race, and location. These encoded biases can influence the decision of a recommendation system (RS) towards further separation of the contents provided to various demographic subgroups, and raise privacy concerns regarding the disclosure of users' protected attributes. In this work, we investigate the possibility and challenges of removing specific protected information of users from the learned interaction representations of a RS algorithm, while maintaining its effectiveness. Specifically, we incorporate adversarial training into the state-of-the-art MultVAE architecture, resulting in a novel model, Adversarial Variational Auto-Encoder with Multinomial Likelihood (Adv-MultVAE), which aims at removing the implicit information of protected attributes while preserving recommendation performance. We conduct experiments on the MovieLens-1M and LFM-2b-DemoBias datasets, and evaluate the effectiveness of the bias mitigation method based on the inability of external attackers in revealing the users' gender information from the model. Comparing with baseline MultVAE, the results show that Adv-MultVAE, with marginal deterioration in performance (w.r.t. NDCG and recall), largely mitigates inherent biases in the model on both datasets.

This tutorial aims at providing its audience an interdisciplinary overview about the topics of fairness and non-discrimination, diversity, and transparency of AI systems, tailored to the research fields of information retrieval and recommender systems. By means of this tutorial, we would like to equip the mostly technical audience of SIGIR with the necessary understanding of the ethical implications of their research and development on the one hand, and of recent political and legal regulations that address the aforementioned challenges on the other hand.

In this article, we report on the creation of a publicly available, common evaluation framework for Violent Scenes Detection (VSD) in Hollywood and YouTube videos. We propose a robust data set, the VSD96, with more than 96 hours of video of various genres, annotations at different levels of detail (e.g., shot-level, segment-level), annotations of mid-level concepts (e.g., blood, fire), various pre-computed multi-modal descriptors, and over 230 system output results as baselines. This is the most comprehensive data set available to this date tailored to the VSD task and was extensively validated during the MediaEval benchmarking campaigns. Furthermore, we provide an in-depth analysis of the crucial components of VSD algorithms, by reviewing the capabilities and the evolution of existing systems (e.g., overall trends and outliers, the influence of the employed features and fusion techniques, the influence of deep learning approaches). Finally, we discuss the possibility of going beyond state-of-the-art performance via an ad-hoc late fusion approach. Experimentation is carried out on the VSD96 data. We provide the most important lessons learned and gained insights. The increasing number of publications using the VSD96 data underline the importance of the topic. The presented and published resources are a practitioner's guide and also a strong baseline to overcome, which will help researchers for the coming years in analyzing aspects of audio-visual affect and violence detection in movies and videos.

Homophily describes the phenomenon that similarity breeds connection, i.e., individuals tend to form ties with other people who are similar to themselves in some aspect(s). The similarity in music taste can undoubtedly influence who we make friends with and shape our social circles. In this paper, we study homophily in an online music platform Last.fm regarding user preferences towards listening to mainstream (M), novel (N), or diverse (D) content. Furthermore, we draw comparisons with homophily based on listening profiles derived from artists users have listened to in the past, i.e., artist profiles. Finally, we explore the utility of users' artist profiles as well as features describing M, N, and D for the task of link prediction. Our study reveals that: (i) users with a friendship connection share similar music taste based on their artist profiles; (ii) on average, a measure of how diverse is the music two users listen to is a stronger predictor of friendship than measures of their preferences towards mainstream or novel content, i.e., homophily is stronger for D than for M and N; (iii) some user groups such as high-novelty-seekers (explorers) exhibit strong homophily, but lower than average artist profile similarity; (iv) using M, N and D achieves comparable results on link prediction accuracy compared with using artist profiles, but the combination of features yields the best accuracy results, and (v) using combined features does not add value if graph-based features such as common neighbors are available, making M, N, and D features primarily useful in a cold-start user recommendation setting for users with few friendship connections. The insights from this study will inform future work on social context-aware music recommendation, user modeling, and link prediction.

Several studies have identified discrepancies between the popularity of items in user profiles and the corresponding recommendation lists. Such behavior, which concerns a variety of recommendation algorithms, is referred to as popularity bias. Existing work predominantly adopts simple statistical measures, such as the difference of mean or median popularity, to quantify popularity bias. Moreover, it does so irrespective of user characteristics other than the inclination to popular content. In this work, in contrast, we propose to investigate popularity differences (between the user profile and recommendation list) in terms of median, a variety of statistical moments, as well as similarity measures that consider the entire popularity distributions (Kullback-Leibler divergence and Kendall’s τ rank-order correlation). This results in a more detailed picture of the characteristics of popularity bias. Furthermore, we investigate whether such algorithmic popularity bias affects users of different genders in the same way. We focus on music recommendation and conduct experiments on the recently released standardized LFM-2b dataset, containing listening profiles of Last.fm users. We investigate the algorithmic popularity bias of seven common recommendation algorithms (five collaborative filtering and two baselines). Our experiments show that (1) the studied metrics provide novel insights into popularity bias in comparison with only using average differences, (2) algorithms less inclined towards popularity bias amplification do not necessarily perform worse in terms of utility (NDCG), (3) the majority of the investigated recommenders intensify the popularity bias of the female users.

Providing suitable recommendations is of vital importance to improve the user satisfaction of music recommender systems. Here, users often listen to the same track repeatedly and appreciate recommendations of the same song multiple times. Thus, accounting for users’ relistening behavior is critical for music recommender systems. In this paper, we describe a psychology-informed approach to model and predict music relistening behavior that is inspired by studies in music psychology, which relate music preferences to human memory. We adopt a well-established psychological theory of human cognition that models the operations of human memory, i.e., Adaptive Control of Thought—Rational (ACT-R). In contrast to prior work, which uses only the base-level component of ACT-R, we utilize five components of ACT-R, i.e., base-level, spreading, partial matching, valuation, and noise, to investigate the effect of five factors on music relistening behavior: (i) recency and frequency of prior exposure to tracks, (ii) co-occurrence of tracks, (iii) the similarity between tracks, (iv) familiarity with tracks, and (v) randomness in behavior. On a dataset of 1.7 million listening events from Last.fm, we evaluate the performance of our approach by sequentially predicting the next track(s) in user sessions. We find that recency and frequency of prior exposure to tracks is an effective predictor of relistening behavior. Besides, considering the co-occurrence of tracks and familiarity with tracks further improves performance in terms of R-precision. We hope that our work inspires future research on the merits of considering cognitive aspects of memory retrieval to model and predict complex user behavior.

Podcasts are spoken documents across a wide-range of genres and styles, with growing listenership across the world, and a rapidly lowering barrier to entry for both listeners and creators. The great strides in search and recommendation in research and industry have yet to see impact in the podcast space, where recommendations are still largely driven by word of mouth. In this perspective paper, we highlight the many differences between podcasts and other media, and discuss our perspective on challenges and future research directions in the domain of podcast information access.

Societal biases resonate in the retrieved contents of information retrieval (IR) systems, resulting in reinforcing existing stereotypes. Approaching this issue requires established measures of fairness in respect to the representation of various social groups in retrieval results, as well as methods to mitigate such biases, particularly in the light of the advances in deep ranking models. In this work, we first provide a novel framework to measure the fairness in the retrieved text contents of ranking models. Introducing a ranker-agnostic measurement, the framework also enables the disentanglement of the effect on fairness of collection from that of rankers. To mitigate these biases, we propose AdvBert, a ranking model achieved by adapting adversarial bias mitigation for IR, which jointly learns to predict relevance and remove protected attributes. We conduct experiments on two passage retrieval collections (MSMARCO Passage Re-ranking and TREC Deep Learning 2019 Passage Re-ranking), which we extend by fairness annotations of a selected subset of queries regarding gender attributes. Our results on the MSMARCO benchmark show that, (1) all ranking models are less fair in comparison with ranker-agnostic baselines, and (2) the fairness of Bert rankers significantly improves when using the proposed AdvBert models. Lastly, we investigate the trade-off between fairness and utility, showing that we can maintain the significant improvements in fairness without any significant loss in utility.

Click logs are valuable resources for a variety of information retrieval (IR) tasks. This includes query understanding/analysis, as well as learning effective IR models particularly when the models require large amounts of training data. We release a large-scale domain-specific dataset of click logs, obtained from user interactions of the Trip Database health web search engine. Our click log dataset comprises approximately 5.2 million user interactions collected between 2013 and 2020. We use this dataset to create a standard IR evaluation benchmark - TripClick - with around 700,000 unique free-text queries and 1.3 million pairs of query-document relevance signals, whose relevance is estimated by two click-through models. As such, the collection is one of the few datasets offering the necessary data richness and scale to train neural IR models with a large amount of parameters, and notably the first in the health domain. Using TripClick, we conduct experiments to evaluate a variety of IR models, showing the benefits of exploiting this data to train neural architectures. In particular, the evaluation results show that the best performing neural IR model significantly improves the performance by a large margin relative to classical IR models, especially for more frequent queries.

Although recommender systems (RSs) play a crucial role in our society, previous studies have revealed that the performance of RSs may considerably differ between groups of individuals with different characteristics or from different demographics. In this case, a RS is considered to be unfair when it does not perform equally well for different groups of users. Considering the importance of RSs in the distribution and consumption of musical content worldwide, a careful evaluation of fairness in the context of music RSs is crucial. To this end, we first introduce LFM-2b, a novel large-scale real-world dataset of music listening records, comprising a subset to investigate bias of RSs regarding users’ demographics. We then define a notion of fairness based on the performance gap of a RS between the users with different demographics, and evaluate a variety of collaborative filtering algorithms in terms of accuracy and beyond-accuracy metrics to explore the fairness in the RS results toward a specific gender group. We observe the existence of significant discrepancies (unfairness) between the performance of algorithms across male and female user groups. Based on these discrepancies, we explore to what extent recommender algorithms lead to intensifying the underlying population bias in the final results. We also study the effect of a resampling strategy, commonly used as debiasing method , which yields slight improvements in the fairness measures of various algorithms while maintaining their accuracy and beyond-accuracy performance.

We present Music Tower Blocks, a novel browsing interface for interactive music visualization, capable of dealing with web-scale music collections nowadays offered by major music streaming services. Based on a clustering created from fused metadata and acoustic features, a block-based skyline landscape is constructed. It can be navigated by the user in several ways (zooming, panning, changing angle of slope). User-adjustable color coding is used for highlighting various facets, e.g., visualizing the distributions of genres and acoustic features. Furthermore, several search and filtering capabilities are provided (e.g., search for artists and tracks; filtering with respect to track popularity to focus on top hits or discovering unknown gems). In addition, Music Tower Blocks offers the user to connect to their personal music streaming profiles and highlight on the landscape their favorite or recently-listened-to music, to support exploring parts of the landscape near to (or far-away from) their own taste.

Recommender systems, like other tools that make use of machine learning, are known to create or increase certain biases. Earlier work has already unveiled different performance of recommender systems for different user groups, depending on gender, age, country, and consumption behavior. In this work, we study user bias in terms of another aspect, i.e., users’ personality. We investigate to which extent state-of-the-art recommendation algorithms yield different accuracy scores depending on the users’ personality traits. We focus on the music domain and create a dataset of Twitter users’ music consumption behavior and personality traits, measuring the latter in terms of the OCEAN model. Investigating [email protected] and [email protected] of the recommendation algorithms SLIM, embarrassingly shallow autoencoders for sparse data (EASE), and variational autoencoders for collaborative filtering (Mult-VAE) on this dataset, we find several significant differences in performance between user groups scoring high vs. groups scoring low on several personality traits.

Concerns regarding the footprint of societal biases in information retrieval (IR) systems have been raised in several previous studies. In this work, we examine various recent IR models from the perspective of the degree of gender bias in their retrieval results. To this end, we first provide a bias measurement framework which includes two metrics to quantify the degree of the unbalanced presence of gender-related concepts in a given IR model's ranking list. To examine IR models by means of the framework, we create a dataset of non-gendered queries, selected by human annotators. Applying these queries to the MS MARCO Passage retrieval collection, we then measure the gender bias of a BM25 model and several recent neural ranking models. The results show that while all models are strongly biased toward male, the neural models, and in particular the ones based on contextualized embedding models, significantly intensify gender bias. Our experiments also show an overall increase in the gender bias of neural models when they exploit transfer learning, namely when they use (already biased) pre-trained embeddings.

Past studies have shown that personality has a significant association with user behaviour and preferences, not least towards music. This makes personality information a promising aspect for user modelling in personalised recommender systems and similar domains. In contrast to existing studies, which investigate personality correlates of music preferences via genres or styles, we study such correlates by modelling music preferences at a finer-grained content level, using audio features of the music users listen to. Leveraging listening and personality information of more than 1,300 Last.fm users, we identify several significant medium and weak correlations between music audio features and personality traits, the latter defined by the five-factor model. Our results provide useful insights into the relationship between personality and music preference, which can be valuable for music recommender systems in terms of more personalised recommendations.

Multimedia search is an emerging area in information retrieval (IR) and recommender systems (RS) research. However, there is a lack of standardized audiovisual datasets that include rich content descriptors, which are a necessity in content-based IR and RS. The contributions of this paper are twofold: First, we present a new multimedia dataset of movie clips, named MFVCD-7K Multifaceted Video Clip Dataset, that comes with low-level and semantic multimodal descriptions of their content (textual, audio, and visual). In addition, we showcase the use of this dataset for a novel content-based video clip retrieval and result diversification task we introduce. We investigate baseline algorithms for retrieval and diversification, and provide experimental results according to relevance and diversity measures. We believe that both dataset and baseline results constitute an important asset for the IR, RS, and multimedia communities.

We present a demo application of a web-based recommender systems that is powered by the “Movie Genome”, i.e., a rich semantic description of a movie's content, including state-of-the-art audio and visual descriptors and metadata (genre and tags). The current version of the Movie Genome web application implements content-based filtering approaches. Due to its modular implementation and the free availability of its source code, it can be easily extended to various context-aware and user-aware scenarios. A personality questionnaire is already integrated into the web application, which allows it to also serve as testbed for personality-aware recommendation algorithms.

We present a browsing interface that allows for an audiovisual exploration of regional music taste around the world. We exploit a total of 10,758,121 geolocated tweets about music. The web-based geo-aware visualization and auralization called Tastalyzer enables exploring and analyzing music taste on a fine-grained geographical level, such as (i) comparing rural and corresponding urban music taste within an agglomeration (city) or (ii) comparing the music taste in a target region (agglomeration) to the taste of the country the region is part of and (iii) to the global music taste.

Music is known to exhibit different characteristics, depending on genre and style. While most research that studies such differences takes a musicological perspective and analyzes acoustic properties of individual pieces or artists, we conduct a large-scale analysis using various web resources. Exploiting content information from song lyrics, contextual information reflected in music artists' Wikipedia articles, and listening information, we particularly study the aspects of popularity, length, repetitiveness, and readability of lyrics and Wikipedia articles. We measure popularity in terms of song play count (PC) and listener count (LC), length in terms of character and word count, repetitiveness in terms of text compression ratio, and readability in terms of the Simple Measure of Gobbledygook (SMOG). Extending datasets of music listening histories and genre annotations from Last.fm, we extract and analyze 424,476 song lyrics by 18,724 artists from LyricWiki.

Music streaming services increasingly incorporate different ways for users to browse for music. Next to the commonly used “genre” taxonomy, nowadays additional taxonomies, such as mood and activities, are often used. As additional taxonomies have shown to be able to distract the user in their search, we looked at how to predict taxonomy preferences in order to counteract this. Additionally, we looked at how the number of categories presented within a taxonomy influences the user experience. We conducted an online user study where participants interacted with an application called “Tune-A-Find”. We measured taxonomy choice (i.e., mood, activity, or genre), individual differences (e.g., personality traits and music expertise factors), and different user experience factors (i.e., choice difficulty and satisfaction, perceived system usefulness and quality) when presenting either 6- or 24-categories within the picked taxonomy. Among 297 participants, we found that personality traits are related to music taxonomy preferences. Furthermore, our findings show that the number of categories within a taxonomy influences the user experience in different ways and is moderated by music expertise. Our findings can support personalized user interfaces in music streaming services. By knowing the user’s personality and expertise, the user interface can adapt to the user’s preferred way of music browsing and thereby mitigate the problems that music listeners are facing while finding their way through the abundance of music choices online nowadays.

Online activities such as social networking, online shopping, and consuming multi-media create digital traces, which are often analyzed and used to improve user experience and increase revenue, e. g., through better-fitting recommendations and more targeted marketing. Analyses of digital traces typically aim to find user traits such as age, gender, and nationality to derive common preferences. We investigate to which extent the music listening habits of users of the social music platform Last.fm can be used to predict their age, gender, and nationality. We propose a feature modeling approach building on Term Frequency-Inverse Document Frequency (TF-IDF) for artist listening information and artist tags combined with additionally extracted features. We show that we can substantially outperform a baseline majority voting approach and can compete with existing approaches. Further, regarding prediction accuracy vs. available listening data we show that even one single listening event per user is enough to outperform the baseline in all prediction tasks. We also compare the performance of our algorithm for different user groups and discuss possible prediction errors and how to mitigate them. We conclude that personal information can be derived from music listening information, which indeed can help better tailoring recommendations, as we illustrate with the use case of a music recommender system that can directly utilize the user attributes predicted by our algorithm to increase the quality of it’s recommendations.

The ACM Recommender Systems Challenge 2018 focused on the task of automatic music playlist continuation, which is a form of the more general task of sequential recommendation. Given a playlist of arbitrary length with some additional meta-data, the task was to recommend up to 500 tracks that fit the target characteristics of the original playlist. For the RecSys Challenge, Spotify released a dataset of one million user-generated playlists. Participants could compete in two tracks, i.e., main and creative tracks. Participants in the main track were only allowed to use the provided training set, however, in the creative track, the use of external public sources was permitted. In total, 113 teams submitted 1,228 runs to the main track; 33 teams submitted 239 runs to the creative track. The highest performing team in the main track achieved an R-precision of 0.2241, an NDCG of 0.3946, and an average number of recommended songs clicks of 1.784. In the creative track, an R-precision of 0.2233, an NDCG of 0.3939, and a click rate of 1.785 was obtained by the best team. This article provides an overview of the challenge, including motivation, task definition, dataset description, and evaluation. We further report and analyze the results obtained by the top-performing teams in each track and explore the approaches taken by the winners. We finally summarize our key findings, discuss generalizability of approaches and results to domains other than music, and list the open avenues and possible future directions in the area of automatic playlist continuation.

As of today, most movie recommendation services base their recommendations on collaborative filtering (CF) and/or content-based filtering (CBF) models that use metadata (e.g., genre or cast). In most video-on-demand and streaming services, however, new movies and TV series are continuously added. CF models are unable to make predictions in such a scenario, since the newly added videos lack interactions—a problem technically known as new item cold start (CS). Currently, the most common approach to this problem is to switch to a purely CBF method, usually by exploiting textual metadata. This approach is known to have lower accuracy than CF because it ignores useful collaborative information and relies on human-generated textual metadata, which are expensive to collect and often prone to errors. User-generated content, such as tags, can also be rare or absent in CS situations. In this paper, we introduce a new movie recommender system that addresses the new item problem in the movie domain by (i) integrating state-of-the-art audio and visual descriptors, which can be automatically extracted from video content and constitute what we call the movie genome ; (ii) exploiting an effective data fusion method named canonical correlation analysis , which was successfully tested in our previous works Deldjoo et al. (in: International Conference on Electronic Commerce and Web Technologies. Springer, Berlin, pp 34–45, 2016b ; Proceedings of the Twelfth ACM Conference on Recommender Systems. ACM, 2018b ), to better exploit complementary information between different modalities; (iii) proposing a two-step hybrid approach which trains a CF model on warm items (items with interactions) and leverages the learned model on the movie genome to recommend cold items (items without interactions). Experimental validation is carried out using a system-centric study on a large-scale, real-world movie recommendation dataset both in an absolute cold start and in a cold to warm transition; and a user-centric online experiment measuring different subjective aspects, such as satisfaction and diversity. Results show the benefits of this approach compared to existing approaches.

Music recommender systems have become a key technology to support the interaction of users with the increasingly larger music catalogs of on-line music streaming services, on-line music shops, and personal devices. An important task in music recommender systems is the automated continuation of music playlists, that enables the recommendation of music streams adapting to given (possibly short) listening sessions. Previous works have shown that applying collaborative filtering to collections of curated music playlists reveals underlying playlist-song co-occurrence patterns that are useful to predict playlist continuations. However, most music collections exhibit a pronounced long-tailed distribution. The majority of songs occur only in few playlists and, as a consequence, they are poorly represented by collaborative filtering. We introduce two feature-combination hybrid recommender systems that extend collaborative filtering by integrating the collaborative information encoded in curated music playlists with any type of song feature vector representation. We conduct off-line experiments to assess the performance of the proposed systems to recover withheld playlist continuations, and we compare them to competitive pure and hybrid collaborative filtering baselines. The results of the experiments indicate that the introduced feature-combination hybrid recommender systems can more accurately predict fitting playlist continuations as a result of their improved representation of songs occurring in few playlists.

Social media represents a valuable data source for researchers to analyze how people feel about a variety of topics, from politics to products to entertainment. This paper addresses the detection of controversies involving music artists, based on microblogs. In particular, we develop a new controversy detection dataset consisting of 53,441 tweets related to 95 music artists, and we devise and evaluate a comprehensive set of user-and content-based feature candidates to regress controversy. The evaluation results show a strong performance of the presented approach in the controversy detection task: F1 score of 0.811 in a classification task and RMSE of 0.688 in a regression task, using controversy scores in the range [1, 4]. In addition, the results obtained in applying the presented approach on a dataset from a different domain (CNN news controversy) demonstrate transferability of the developed feature set, with a significant improvement over prior approaches. A combination of the adopted Gradient Boosting based classifier and the developed feature set results in an F1 score of 0.775, which represents an improvement of 9.8% compared to the best prior result on this dataset.

In this paper we propose a new dataset, i.e., the MMTF-14K multi-faceted dataset. It is primarily designed for the evaluation of video-based recommender systems, but it also supports the exploration of other multimedia tasks such as popularity prediction, genre classification and auto-tagging (aka tag prediction). The data consists of 13,623 Hollywood-type movie trailers, ranked by 138,492 users, generating a total of almost 12.5 million ratings. To address a broader community, metadata, audio and visual descriptors are also pre-computed and provided along with several baseline benchmarking results for uni-modal and multi-modal recommendation systems. This creates a rich collection of data for benchmarking results and which supports future development of this field.

The ACM Recommender Systems Challenge 2018 focused on automatic music playlist continuation, which is a form of the more general task of sequential recommendation. Given a playlist of arbitrary length, the challenge was to recommend up to 500 tracks that fit the target characteristics of the original playlist. For the Challenge, Spotify released a dataset of one million user-created playlists, along with associated metadata. Participants could submit their approaches in two tracks, i.e., main and creative tracks, where the former allowed teams to use solely the provided dataset and the latter allowed them to exploit publicly available external data too. In total, 113 teams submitted 1,228 runs in the main track; 33 teams submitted 239 runs in the creative track. The highest performing team in the main track achieved an R-precision of 0.2241, an NDCG of 0.3946, and an average number of recommended songs clicks of 1.784. In the creative track, an R-precision of 0.2233, an NDCG of 0.3939, and a click rate of 1.785 was realized by the best team.

We propose a multi-modal content-based movie recommender system that replaces human-generated metadata with content descriptions automatically extracted from the visual and audio channels of a video. Content descriptors improve over traditional metadata in terms of both richness (it is possible to extract hundreds of meaningful features covering various modalities) and quality (content features are consistent across different systems and immune to human errors). Our recommender system integrates state-of-the-art aesthetic and deep visual features as well as block-level and i-vector audio features. For fusing the different modalities, we propose a rank aggregation strategy extending the Borda count approach.

This tutorial introduces multimedia recommender systems (MMRS), in particular, recommender systems that leverage multimedia content to recommend different media types. In contrast to the still most frequently adopted collaborative filtering approaches, we focus on content-based MMRS and on hybrids of collaborative filtering and content-based filtering. The target recommendation domains of the tutorial are movies, music and images. We present state-of-the-art approaches for multimedia feature extraction (text, audio, visual), including deep learning methods, and recommendation approaches tailored to the multimedia domain. Furthermore, by introducing common evaluation techniques, pointing to publicly available datasets specific to the multimedia domain, and discussing the grand challenges in MMRS research, this tutorial provides the audience with a profound introduction to MMRS and an inspiration to conduct further research.

Integrating information about the listener's cultural background when building music recommender systems has recently been identified as a means to improve recommendation quality. In this paper, we therefore propose a novel approach to jointly model users by the user's musical preferences and his/her cultural background. We describe the musical preferences of users by relying on the acoustic features of the songs the users have been listening to and characterize the cultural background of users by cultural and socio-economic features that we infer from the user's country. To evaluate the impact of the proposed user model on recommendation quality, we integrate the model into a culture-aware music recommender system. We show that incorporating both acoustic information of the tracks a user has listened to as well as the cultural background of users in the form of a music-cultural user model contributes to improved recommendation performance.

This study deals with the strong relationship between emotions and music, investigating three main research questions: (RQ1) Are there differences in human music perception (e.g., emotions, tempo, instrumentation, and complexity), according to musical education, experience, demographics, and personality traits?; (RQ2) Do certain perceived music characteristics correlate (e.g., tension and sadness), irrespective of a particular listener's background or personality?; (RQ3) Does human perception of music characteristics, such as emotions and tempo, correlate with descriptors extracted from music audio signals? To investigate our research questions, we conducted two user studies focusing on different groups of subjects. We used web-based surveys to collect information about demographics, listening habits, musical education, personality, and perceptual ratings with respect to perceived emotions, tempo, complexity, and instrumentation for 15 segments of Beethoven's 3rd symphony, “Eroica”. Our experiments showed that all three research questions can be affirmed, at least partly. We found strong support for RQ2 and RQ3, while RQ1 could be confirmed only for some perceptual aspects and user groups.

Incorporating user characteristics and contextual information has shown to be essential when it comes to personalized music retrieval and recommendation. To this end, the current location of a user is often exploited. However, relying solely on GPS coordinates neglects the cultural background of users, which does not necessarily coincide with political borders. In this paper, we analyze culture-specific music listening behavior based on a dataset of 2,724 Spotify users, 62,104 distinct tracks and 104,390 listening events by modeling users jointly via their musical preferences and cultural characteristics. By applying a density-based spatial clustering algorithm, we identify nine clusters which reflect similar users regarding both their musical preference and cultural background. Our findings show that cultural aspects cannot be approximated by GPS coordinates and that incorporating cultural characteristics allows for more precise user characterization. Also, we observe that listening patterns occur on two different levels: we observe country-specific listening patterns as well as cross-country listening patterns that span across several countries.

In this paper, we describe the LFM-1b User Genre Profile dataset. It provides detailed information on musical genre preferences for more than 120,000 listeners and links to the LFM-1b dataset. We created the dataset by exploiting social tags, indexing them using two genre term sets, and aggregating the resulting annotated listening events on the user level. We foresee several applications of the dataset in music retrieval and recommendation tasks, among others to build and evaluate decent user models, to alleviate cold-start situations in music recommender systems, and to increase their performance using the additional abstraction layer of genre. We further present results of statistical analyses of the dataset, regarding genre preferences and their consistencies. We do so for the entire user population and for user groups defined by demographic similarities. Moreover, we report interesting insights about correlations between musical preferences on the genre level.

Considering the cultural background of users is known to improve recommender systems for multimedia items. In this work, we focus on music and analyze user demographics and music listening events in a large corpus (120,000 users, 109 events) from Last.fm to investigate whether similarity between countries in terms of cultural and socio-economic factors is reflected in music taste. To this end, we propose a tag-based model to describe the music taste of a country and correlate the resulting music profiles to Hofstede's cultural dimensions and the Quality of Government data. Spearman's rank-order correlation and Quadratic Assignment Procedure indeed indicate statistically significant weak to medium correlations of music taste and several cultural and socio-economic factors. The results will help elaborating culture-aware models of music listeners and in turn likely yield improved music recommender systems.

The music mainstreaminess of a user reflects how strong a user's listening preferences correspond to those of the larger population. Considering that music mainstream may be defined from different perspectives and on various levels, e.g., geographical (charts of a country), genre ("Indie charts"), or distribution channel (radio charts vs. download charts), we study how the user's music mainstreaminess influences the quality of music recommendations. The paper's contribution is three-fold. First, we propose 11 novel mainstreaminess measures characterizing music listeners, considering both a global and a country-specific basis. To this end, we model preference profiles (as a vector over artists) for users, countries, and globally, incorporating artist frequency, listener frequency, and a newly proposed TF-IDF-inspired weighting function, which we call artist frequency--inverse listener frequency (AF-ILF). The resulting preference profile for each user u is then related to the respective country-specific and global preference profile using fraction-based approaches, symmetrized Kullback-Leibler divergence, and Kendall's τ rank correlation, in order to quantify u's mainstreaminess. Second, we demonstrate country-specific peculiarities of these mainstreaminess definitions. Third, we show that incorporating the proposed global and country-specific mainstreaminess measures into the music recommendation process can notably improve accuracy of rating prediction.

The particularities of musical data and its multiple modalities make original contributions possible in many core RecSys topics such as content-based and hybrid recommendation, user modeling, interfaces, and context-aware and mobile recommendations. But more urgently, the current revolution in the music industry represents major opportunities and challenges for recommendation systems in general. Recommendation systems are now central to music streaming platforms, which are rapidly increasing in listenership and becoming the top source of revenue for the music industry. It is increasingly more common for a music listener to simply access music than to purchase and own it in a personal collection. In this scenario, recommendation calls no longer for a one-shot recommendation for the purpose of a track or album purchase, but for a recommendation of a listening experience, comprising a very wide range of challenges, such as sequential recommendation, or conversational and contextual recommendations. Recommendation technologies now impact all actors in the rich and complex music industry ecosystem (listeners, labels, music makers and producers, concert halls, advertisers, etc.). To highlight these developments, we focus on three use cases: automatic playlist generation, context-aware music recommendation, and recommendation in the creative process of music making.

Personality-based personalized systems are increasingly gaining interest as personality traits has been shown to be a stable construct within humans. In order to provide a personality-based experience to the user, users' behavior, preferences, and needs in relation to their personality need to be investigated. Although for a technological mediated environment the search for these relationships is often new territory, there are findings from personality research of the real world that can be used in personalized systems. However, for these findings to be implementable, we need to investigate whether they hold in a technologically mediated environment. In this study we assess prior work on personality-based music genre preferences from traditional personality research. We analyzed a dataset consisting of music listening histories and personality scores of 1415 Last.fm users. Our results show agreements with prior work, but also important differences that can help to inform personalized systems.

Intelligent technologies have become an essential part of music systems and applications. This is evidenced by today's omnipresence of digital online music stores and streaming services, which rely on music recommenders, automatic playlist generators, and music browsing interfaces. A large amount of research leading to intelligent music applications deals with the extraction of musical and acoustic information directly from the audio signal using signal processing techniques. Other strategies exploit contextual aspects of music, not present in the signal, for example, community meta-data and trails of user interaction, as found, for instance, on social media platforms. In this editorial, we discuss the notion of “intelligent music system” and give an overview of the papers selected to this special issue.

We present the LFM-1b dataset of more than one billion music listening events created by more than 120,000 users of Last.fm. Each listening event is characterized by artist, album, and track name, and further includes a timestamp. On the (anonymous) user level, basic demographics and a selection of more elaborate user descriptors are included.

This paper reports on three demonstrators developed to provide personalized, enhanced experiences of classical music, within the EU FP7 project "Performances as Highly Enriched aNd Interactive Concert eXperiences" (PHENICX): (i) an interface for accessing supplemental multimodal material about music items, (ii) a recommender system for visualizations of classical music, and (iii) a recommender system for tagging. The personalization in all three demos is achieved through modeling users' personality and musical sophistication. The links to the web interfaces are provided as well as the outcomes of quantitative and qualitative evaluations.

To enhance the experience of listening to classical orchestra music, either in the concert hall or at home, we present a personalized system that integrates three visualization/interaction concepts: Score Follower (points to the current position in the score), Orchestra Layout (illustrates instruments that are currently playing and their dynamics), and Structure Visualization (visualizes structural elements such as themes or motifs). Motivated by previous literature that found evidence for connections between personality and music consumption and preference, we first assessed in a user study to which extent personality traits and music visualization preferences correlate. Measuring preference via pragmatic quality and personality traits according to the Big Five Inventory (BFI) questionnaire, we found substantial interconnections between them. These translate into rules relating certain personality traits (e.g., extraversion or agreeableness) to preference rankings of the visualizations.

Providing diversity in recommendations has shown to positively influence the user's subjective evaluations such as satisfaction. However, it is often unknown how much diversity a recommendation set needs to consist of. In this work, we explored how music users of Last.fm apply diversity in their listening behavior. We analyzed a dataset with the music listening history of 53,309 Last.fm users capturing their total listening events until August 2014. We complemented this dataset with The Echo Nest features and Hofstede's cultural dimensions to explore how music diversity is applied across countries. Between 47 countries, we found distinct relationships between the cultural dimensions and music diversity variables. These results suggest that different country-based diversity measurements should be considered when applied to a recommendation set in order to maximize the user's subjective evaluations. The country-based relationships also provide opportunities for recommender systems to personalize experiences when user data is limited by being able to rely on the user's demographics.

In this tutorial, we give an introduction to the field of and state of the art in music information retrieval (MIR). The tutorial particularly spotlights the question of music similarity, which is an essential aspect in music retrieval and recommendation. Three factors play a central role in MIR research: (1) the music content, i.e., the audio signal itself, (2) the music context, i.e., metadata in the widest sense, and (3) the listeners and their contexts, manifested in user-music interaction traces. We review approaches that extract features from all three data sources and combinations thereof and show how these features can be used for (large-scale) music indexing, music description, music similarity measurement, and recommendation. These methods are further showcased in a number of popular music applications, such as automatic playlist generation and personalized radio stationing, location-aware music recommendation, music search engines, and intelligent browsing interfaces. Additionally, related topics such as music identification, automatic music accompaniment and score following, and search and retrieval in the music production domain are discussed.

A shortcoming of current approaches for music recommendation is that they consider user-specific characteristics only on a very simple level, typically as some kind of interaction between users and items when employing collaborative filtering. To alleviate this issue, we propose several user features that model aspects of the user's music listening behavior: diversity, mainstreaminess, and novelty of the user's music taste. To validate the proposed features, we conduct a comprehensive evaluation of a variety of music recommendation approaches (stand-alone and hybrids) on a collection of almost 200 million listening events gathered from \propername{Last.fm}. We report first results and highlight cases where our diversity, mainstreaminess, and novelty features can be beneficially integrated into music recommender systems.

The amount of music consumed while on the move has been spiraling during the past couple of years, which requests for intelligent music recommendation techniques. In this demo paper, we introduce a context-aware mobile music player named "Mobile Music Genius" (MMG), which seamlessly adapts the music playlist on the fly, according to the user context. It makes use of a comprehensive set of features derived from sensor data, spatiotemporal information, and user interaction to learn which kind of music a listeners prefers in which context. We describe the automatic creation and adaptation of playlists and present results of a study that investigates the capabilities of the gathered user context features to predict the listener's music preference.

The SoMeRA workshop targets cutting edge research from all fields of retrieval, recommendation, and browsing in social media, as well as the analysis of user's multifaceted traces therein. Submissions to the workshop cover a broad range of topics including multimedia retrieval and exploration, user-aware recommender systems, network analysis, event detection, and computational linguistics.

Music information retrieval and music recommendation are seeing a paradigm shift towards methods that incorporate user context aspects. However, structured experiments on a standardized music dataset to investigate the effects of doing so are scarce. In this paper, we compare performance of various combinations of collaborative filtering and geospatial as well as cultural user models for the task of music recommendation. To this end, we propose a geospatial model that uses GPS coordinates and a cultural model that uses semantic locations (continent, country, and state of the user). We conduct experiments on a novel standardized music collection, the ``Million Musical Tweets Dataset'' of listening events extracted from microblogs. Overall, we find that modeling listeners' location via Gaussian mixture models and computing similarities from these outperforms both cultural user models and collaborative filtering.

Microtexts are a valuable, albeit noisy, source to infer collaborative information. As music plays an important role in many human lives, microblogs on music-related activities are available in abundance. This paper investigates different strategies to estimate music similarity from these data sources. In particular, we first present a framework to extract co-occurrence scores between music artists from microblogs and then investigate 12 similarity estimation functions to subsequently derive resemblance scores. We evaluate the approaches on a collection of microblogs crawled from Twitter over a period of 10 months and compare them to standard tf-idf approaches. As evaluation criteria we use precision and recall in an artist retrieval task as well as rank proximity. We show that collaborative chatter on music can be effectively used to develop music artist similarity measures, which are a core part of every music retrieval and recommendation system. Furthermore, we analyze the effects of the “long tail” on retrieval results and investigate whether results are consistent over time, using a second dataset.

In this paper we approach the issue of violence detection in typical Hollywood productions. Given the high variability in appearance of violent scenes in movies, training a classifier to predict violent frames directly from visual or/and auditory features seems rather difficult. Instead, we propose a different perspective that relies on fusing mid-level concept predictions that are inferred from low-level features. This is achieved by employing a bank of multi-layer perceptron classifiers featuring a dropout training scheme. Experimental validation conducted in the context of the Violent Scenes Detection task of the MediaEval 2012 Multimedia Benchmark Evaluation show the potential of this approach that ranked first among 34 other submissions in terms of precision and F1-score.

No abstract available.

This paper presents the MusiClef data set, a multimodal data set of professionally annotated music. It includes editorial metadata about songs, albums, and artists, as well as MusicBrainz identifiers to facilitate linking to other data sets. In addition, several state-of-the-art audio features are provided. Different sets of annotations and music context data -- collaboratively generated user tags, web pages about artists and albums, and the annotation labels provided by music experts -- are included too. Versions of this data set were used in the MusiClef evaluation campaigns in 2011 and 2012 for auto-tagging tasks. We report on the motivation for the data set, on its composition, on related sets, and on the evaluation campaigns in which versions of the set were already used. These campaigns likewise represent one use case, i.e. music auto-tagging, of the data set. The complete data set is publicly available for download at http://www.cp.jku.at/musiclef.

Addressing the spiraling amount of music and video consumption via streaming services, in particular on mobile devices, we present a music player application for the Android platform, which employs a hybrid approach to generate a list of track recommendations for a user. We propose and evaluate two different algorithms, namely a content-based algorithm and an approach that exploits social similarity. While the former is based on rhythm features, the latter exploits "related videos" relations from YouTube. We show via a user questionnaire that recommendation results based on content slightly, but statistically significantly, outperform the social approach. Given that full audio content is not available immediately in a streaming environment, however, we suggest a hybrid, dynamic approach to music recommendation.

Successful music recommendation systems need to incorporate information on at least three levels: the music content, the music context, and the user context. The former refers to features derived from the audio signal; the second refers to aspects of the music or artist not encoded in the audio, nevertheless important to human music perception; the third refers to contextual aspects of the user which change dynamically.

We propose a novel approach to context-aware music recommendation - recommending music suited for places of interest (POIs). The suggested hybrid approach combines two techniques -- one based on representing both POIs and music with tags, and the other based on the knowledge of the semantic relations between the two types of items. We show that our approach can be scaled up using a novel music auto-tagging technique and we compare it in a live user study to: two non-hybrid solutions, either based on tags or on semantic relations; and to a context-free but personalized recommendation approach. In the considered scenario, i.e., a situation defined by a context (the POI), we show that personalization (via music preference) is not sufficient and it is important to implement effective adaptation techniques to the user's context. In fact, we show that the users are more satisfied with the recommendations generated by combining the tag-based and knowledge-based context adaptation techniques, which exploit orthogonal types of relations between places and music tracks.

This tutorial serves as an introductory course to the field of and state-of-the-art in music information retrieval (MIR) and in particular to music similarity estimation which is an essential component of music retrieval. Apart from explaining approaches that estimate similarity based on acoustic properties of an audio signal, we review methods that exploit (mostly textual) meta-data from the Web to build representations of music then used for similarity calculation. Additionally, topics such as (large-scale) music indexing, information extraction for music, personalization in music retrieval, and evaluation of MIR systems are addressed.

Recent advances in music retrieval and recommendation algorithms highlight the necessity to follow multimodal approaches in order to transcend limits imposed by methods that solely use audio, web, or collaborative filtering data. In this paper, we propose hybrid music recommendation algorithms that combine information on the music content, the music context, and the user context, in particular, integrating location-aware weighting of similarities. Using state-of-the-art techniques to extract audio features and contextual web features, and a novel standardized data set of music listening activities inferred from microblogs (MusicMicro), we propose several multimodal retrieval functions.

Personalization and context-awareness are highly important topics in research on Intelligent Information Systems. In the fields of Music Information Retrieval (MIR) and Music Recommendation in particular, user-centric algorithms should ideally provide music that perfectly fits each individual listener in each imaginable situation and for each of her information or entertainment needs. Even though preliminary steps towards such systems have recently been presented at the “International Society for Music Information Retrieval Conference” (ISMIR) and at similar venues, this vision is still far away from becoming a reality. In this article, we investigate and discuss literature on the topic of user-centric music retrieval and reflect on why the breakthrough in this field has not been achieved yet. Given the different expertises of the authors, we shed light on why this topic is a particularly challenging one, taking computer science and psychology points of view. Whereas the computer science aspect centers on the problems of user modeling, machine learning, and evaluation, the psychological discussion is mainly concerned with proper experimental design and interpretation of the results of an experiment. We further present our ideas on aspects crucial to consider when elaborating user-aware music retrieval systems.

In this survey article, we give an overview of methods for music similarity estimation and music recommendation based on music context data. Unlike approaches that rely on music content and have been researched for almost two decades, music-context -based (or contextual ) approaches to music retrieval are a quite recent field of research within music information retrieval (MIR). Contextual data refers to all music-relevant information that is not included in the audio signal itself. In this article, we focus on contextual aspects of music primarily accessible through web technology. We discuss different sources of context-based data for individual music pieces and for music artists. We summarize various approaches for constructing similarity measures based on the collaborative or cultural knowledge incorporated into these data sources. In particular, we identify and review three main types of context-based similarity approaches: text-retrieval-based approaches (relying on web-texts, tags, or lyrics), co-occurrence-based approaches (relying on playlists, page counts, microblogs, or peer-to-peer-networks), and approaches based on user ratings or listening habits. This article elaborates the characteristics of the presented context-based measures and discusses their strengths as well as their weaknesses.

In this paper a new approach for polyphonic piano note onset transcription is presented. It is based on a recurrent neural network to simultaneously detect the onsets and the pitches of the notes from spectral features. Long Short-Term Memory units are used in a bidirectional neural network to model the context of the notes. The use of a single regression output layer instead of the often used one-versus-all classification approach enables the system to significantly lower the number of erroneous note detections. Evaluation is based on common test sets and shows exceptional temporal precision combined with a significant boost in note transcription performance compared to current state-of-the-art approaches. The system is trained jointly with various synthesized piano instruments and real piano recordings and thus generalizes much better than existing systems.

Mobile music consumption has been spiraling during the past couple of years. The interaction techniques provided to sift through the ever increasing amounts of music available on smart devices unfortunately have not. In this paper, we address this issue and present an intelligent mobile user interface that enables the user to browse her mobile music collection in a joyful and informed way.

Different term weighting techniques such as $$TF\cdot IDF$$ or BM25 have been used intensely for manifold text-based information retrieval tasks. Their use for modeling term profiles for named entities and subsequent calculation of similarities between these named entities have been studied to a much smaller extent. The recent trend of microblogging made available massive amounts of information about almost every topic around the world. Therefore, microblogs represent a valuable source for text-based named entity modeling. In this paper, we present a systematic and comprehensive evaluation of different term weighting measures, normalization techniques, query schemes, index term sets, and similarity functions for the task of inferring similarities between named entities, based on data extracted from microblog posts. We analyze several thousand combinations of choices for the above mentioned dimensions, which influence the similarity calculation process, and we investigate in which way they impact the quality of the similarity estimates. Evaluation is performed using three real-world data sets: two collections of microblogs related to music artists and one related to movies. For the music collections, we present results of genre classification experiments using as benchmark genre information from allmusic.com . For the movie collection, we present results of multi-class classification experiments using as benchmark categories from IMDb . We show that microblogs can indeed be exploited to model named entity similarity with remarkable accuracy, provided the correct settings for the analyzed aspects are used. We further compare the results to those obtained when using Web pages as data source.

In this paper, we provide an overview of the 3rd International Workshop on Search and Mining User-generated Contents, held in conjunction with the 20th ACM International Conference on Information and Knowledge Management. We present the motivation and goals of the workshop, and some statistics and details about accepted papers and keynotes.

We present a novel user interface that offers a fun way to explore music collections in virtual landscapes in a game-like manner. Extending previous work, special attention is paid to scalability and user interaction. In this vein, the ever growing size of today's music collections is addressed in two ways that allow for visualizing and browsing nearly arbitrarily sized music repositories. First, the proposed user interface deepTune employs a hierarchical version of the Self-Organizing Map (SOM) to cluster similar pieces of music using multiple, hierarchically aligned layers. Second, to facilitate orientation in the landscape by presenting well-known anchor points to the user, a combination of Web-based and audio signal-based information extraction techniques to determine cluster prototypes (songs) is proposed. Selecting representative and well-known prototypes -- the former is ensured by using signal-based features, the latter by using Web-based data -- is crucial for browsing large music collections. We further report on results of an evaluation carried out to assess the quality of the proposed cluster prototype ranking.

This article comprehensively addresses the problem of similarity measurement between music artists via text-based features extracted from Web pages. To this end, we present a thorough evaluation of different term-weighting strategies, normalization methods, aggregation functions, and similarity measurement techniques. In large-scale genre classification experiments carried out on real-world artist collections, we analyze several thousand combinations of settings/parameters that influence the similarity calculation process, and investigate in which way they impact the quality of the similarity estimates. Accurate similarity measures for music are vital for many applications, such as automated playlist generation, music recommender systems, music information systems, or intelligent user interfaces to access music collections by means beyond text-based browsing. Therefore, by exhaustively analyzing the potential of text-based features derived from artist-related Web pages, this article constitutes an important contribution to context-based music information research.

The origin of a music artist or a band is an important kind of musical meta-data as it usually influences his/her/its music. In this paper, we propose three approaches to automatically determine the country of origin of a person or institution, which we apply to music artists and bands. The first approach investigates estimates of page counts returned for specific queries to Web search engines. The second approach uses term weighting functions for country-specific terms that occur on the top-ranked Web pages of an artist. The third approach applies to Web pages text distance measures between country-specific terms and key terms related to the concept or origin. We further present a thorough evaluation of the approaches taking into consideration different refinements. We show that we are able to outperform the first, nevertheless recent, approach to determine the origin of a music artist.

We propose three heuristics to determine the country of origin of a person or institution via text-based IE from the Web. We evaluate all methods on a collection of music artists and bands, and show that some heuristics outperform earlier work on the topic by terms of coverage, while retaining similar precision levels. We further investigate an extension using country-specific synonym lists.

An approach is presented to automatically build a search engine for large-scale music collections that can be queried through natural language. While existing approaches depend on explicit manual annotations and meta-data assigned to the individual audio pieces, we automatically derive descriptions by making use of methods from Web Retrieval and Music Information Retrieval. Based on the ID3 tags of a collection of mp3 files, we retrieve relevant Web pages via Google queries and use the contents of these pages to characterize the music pieces and represent them by term vectors. By incorporating complementary information about acous tic similarity we are able to both reduce the dimensionality of the vector space and improve the performance of retrieval, i.e. the quality of the results. Furthermore, the usage of audio similarity allows us to also characterize audio pieces when there is no associated information found on the Web.

We present CoMIRVA, which is an abbreviation for Collection of Music Information Retrieval and Visualization Applications. CoMIRVA is a Java framework and toolkit for information retrieval and visualization. It is licensed under the GNU GPL and can be downloaded from http://www.cp.jku.at/comirva/. At the moment, the main functionalities include music information retrieval, web retrieval, and visualization of the extracted information. In this paper, we focus on the visualization aspects of CoMIRVA. Since many of the information retrieval functions are intended to be applied to problems of the field of music information retrieval (MIR), we demonstrate the functions using data like similarity matrices of music artists gained by analyzing artist-related web pages. CoMIRVA is continuously being extended. Currently, it supports the following visualization techniques: Self-Organizing Map, Smoothed Data Histogram, Circled Bars, Circled Fans, Probabilistic Network, Continuous Similarity Ring, Sunburst, and Music Description Map. Since space is limited, we can only present a selected number of these in this paper. As one key feature of CoMIRVA is its easy extensibility, we further elaborate on how CoMIRVA was used for creating a novel user interface to digital music repositories.

We present a novel interface to (portable) music players that benefit from intelligently structured collections of audio files. For structuring, we calculate similarities between every pair of songs and model a travelling salesman problem (TSP) that is solved to obtain a playlist (i.e., the track ordering during playback) where the average distance between consecutive pieces of music is minimal according to the similarity measure. The similarities are determined using both audio signal analysis of the music tracks and Web-based artist profile comparison. Indeed, we show how to enhance the quality of the well-established methods based on audio signal processing with features derived from Web pages of music artists. Using TSP allows for creating circular playlists that can be easily browsed with a wheel as input device. We investigate the usefulness of four different TSP algorithms for this purpose. For evaluating the quality of the generated playlists, we apply a number of quality measures to two real-world music collections. It turns out that the proposed combination of audio and text-based similarity yields better results than the initial approach based on audio data only. We implemented an audio player as Java applet to demonstrate the benefits of our approach. Furthermore, we present the results of a small user study conducted to evaluate the quality of the generated playlists

We present a novel, innovative user interface to music repositories. Given an arbitrary collection of digital music files, our system creates a virtual landscape which allows the user to freely navigate in this collection. This is accomplished by automatically extracting features from the audio signal and training a Self-Organizing Map (SOM) on them to form clusters of similar sounding pieces of music. Subsequently, a Smoothed Data Histogram (SDH) is calculated on the SOM and interpreted as a three-dimensional height profile. This height profile is visualized as a three-dimensional island landscape containing the pieces of music. While moving through the terrain, the closest sounds with respect to the listener's current position can be heard. This is realized by anisotropic auralization using a 5.1 surround sound model. Additionally, we incorporate knowledge extracted automatically from the web to enrich the landscape with semantic information. More precisely, we display words and related images that describe the heard music on the landscape to support the exploration.

A third generation of torque-controlled light weight robots has been developed in DLR's robotics and mechatronics lab which is based on all the experiences that have been had with the first two generations. It aims at reaching the limits of what seems achievable with present day technologies not only with respect to light-weight, but also with respect to minimal power consumption and losses. One of the main gaps we tried to close in version III was the development of a new, robot-dedicated high energy motor designed with the best available techniques of concurrent engineering, and the renewed efforts to save weight in the links by using ultralight carbon fibres.