Whether in a happy mood or a sad mood, humans enjoy listening to music. In this paper, we introduce a novel method to identify auditory features that best predict listener-reported enjoyment ratings by splitting the features into qualitative feature groups, then training predictive models on these feature groups and comparing prediction performance. Using audio features that relate to dynamics, timbre, harmony, and rhythm, we predicted continuous enjoyment ratings for a set of happy and sad songs. We found that a distributed lag model with Ll regularization best predicted these responses and that timbre-related features were most relevant for predicting enjoyment ratings in happy music, while harmony-related features were most relevant to predicting enjoyment ratings in sad music. This work adds to our understanding of how music influences affective human experience.

Music has a powerful influence on human experience. In this paper, we investigate how music affects brain activity, physiological response, and human-reported behavior. Using auditory features related to dynamics, timbre, harmony, rhythm, and register, we predicted brain activity in the form of phase synchronizations in bilateral Heschl's gyri and superior temporal gyri; physiological response in the form of galvanic skin response and heart activity; and emotional experience in the form of continuous, subjective descriptions reported by music listeners. We found that using multivariate time series models with attention mechanisms are effective in predicting emotional ratings, while vector-autoregressive models are effective in predicting involuntary human responses. Musical features related to dynamics, register, rhythm, and harmony were found to be particularly helpful in predicting these human reactions. This work adds to our understanding of how music affects multimodal human experience and has applications in affective computing, music emotion recognition, neuroscience, and music information retrieval.