The important role of the valence and arousal dimensions in representing and recognizing affective qualities in music is well established. There is less evidence for the contribution of secondary dimensions such as potency, tension and energy. In particular, previous studies failed to find significant relations between computable musical features and affective dimensions other than valence and arousal. Here we present two experiments aiming at assessing how musical features, directly computable from complex audio excerpts, are related to secondary emotion dimensions. To this aim, we imposed some constraints on the musical features, namely modality and tempo, of the stimuli.The results show that although arousal and valence dominate for many musical features, it is possible to identify features, in particular Roughness, Loudness, and SpectralFlux, that are significantly related to the potency dimension. As far as we know, this is the first study that gained more insight into the affective potency in the music domain by using real music recordings and a computational approach.

Computational systems for generating expressive musical performances have been studied for several decades now. These models are generally evaluated by comparing their predictions with actual performances, both from a performance parameter and a subjective point of view, often focusing on very specific aspects of the model. However, little is known about how listeners evaluate the generated performances and what factors influence their judgement and appreciation. In this article, we present two studies, conducted during two dedicated workshops, to start understanding how the audience judges entire performances employing different approaches to generating musical expression. In the preliminary study, 40 participants completed a questionnaire in response to five different computer-generated and computer-assisted performances, rating preference and describing the expressiveness of the performances. In the second, “GATM” ( Gruppo di Analisi e Teoria Musicale ) study, 23 participants also completed the Music Cognitive Style questionnaire. Results indicated that music systemizers tend to describe musical expression in terms of the formal aspects of the music, and music empathizers tend to report expressiveness in terms of emotions and characters. However, high systemizers did not differ from high empathizers in their mean preference score across the five pieces. We also concluded that listeners tend not to focus on the basic technical aspects of playing when judging computer-assisted and computer-generated performances. Implications for the significance of individual differences in judging musical expression are discussed.

Expression communication is the added value of a musical performance. It is part of the reason why music is interesting to listen to and sounds alive. Previous work on the analysis of acoustical features yielded relevant features for the recognition of different expressive intentions, inspired both by emotional and sensorial adjectives. In this article, machine learning techniques are employed to understand how expressive performances represented by the selected features are clustered on a low-dimensional space, and to define a measure of acoustical similarity. Being that expressive intentions are similar according to the features used for the recognition, and since recognition implies subjective evaluation, we hypothesized that performances are similar also from a perceptual point of view. We then compared and integrated the clustering of acoustical features with the results of two listening experiments. A first experiment aims at verifying whether subjects can distinguish different categories of expressive intentions, and a second experiment aims at understanding which expressions are perceptually clustered together in order to derive common evaluation criteria adopted by listeners, and to obtain the perceptual organization of affective and sensorial expressive intentions. An interpretation of the resulting spatial representation based on action is proposed and discussed.

We present some results on audio restoration obtained with an algorithm that solves the problems of broadband noise filtering, signal parameters tracking, and impulsive noise removal by using the Extended Kalman Filter (EKF) theory. We show that, to achieve maximum performance, it is essential to optimize the EKF implementation. To this purpose, having to cope with the nonstationarity of the audio signal, we use two properly combined EKF filters (forward and backward), and introduce a bootstrapping procedure for model tracking. The careful combination of the proposed techniques and an accurate choice of some critical parameters, allows to improve the performance of the EKF algorithm. The presented procedure is validated by listening tests.

During a music performance, the musician adds expressiveness to the musical message by changing timing, dynamics, and timbre of the musical events to communicate an expressive intention. Traditionally, the analysis of music expression is based on measurements of the deviations of the acoustic parameters with respect to the written score. In this paper, we employ machine learning techniques to understand the expressive communication and to derive audio features at an intermediate level, between music intended as a structured language and notes intended as sound at a more physical level. We start by extracting audio features from expressive performances that were recorded by asking the musicians to perform in order to convey different expressive intentions. We use a sequential forward selection procedure to rank and select a set of features for a general description of the expressions, and a second one specific for each instrument. We show that higher recognition ratings are achieved by using a set of four features which can be specifically related to qualitative descriptions of the sound by physical metaphors. These audio features can be used to retrieve expressive content on audio data, and to design the next generation of search engines for music information retrieval.

We propose an original technique for separating the spectrum of the noisy component from that of the sinusoidal, quasi-deterministic one, for the sinusoids + transients + noise modeling of musical sounds. It also enables estimation of the time-domain noise envelope and detection of transients with standard techniques. The algorithm for spectrum separation relies on nonlinear transformations of the amplitude spectrum of the sampled signal obtained via fast Fourier transform, which allow to eliminate the dominant partials without the need for precisely tuned notch filters. The envelope estimation is performed by calculating the energy of the signal in the frequency domain, over a sliding time window. Several transformations (such as pitch shifting, time stretching, etc.) can be performed on the so-obtained stochastic spectrum prior to resynthesis. The synthesized sound is built via inverse fast Fourier transform with overlap-add method. The performance of the proposed algorithm is assessed on synthetic, instrumental, and natural sounds in terms of different quality measures

Expression is an important aspect of music performance. It is the added value of a performance and is part of the reason that music is interesting to listen to and sounds alive. Understanding and modeling expressive content communication is important for many engineering applications in information technology. For example, in multimedia products, textual information is enriched by means of graphical and audio objects. In this paper, we present an original approach to modify the expressive content of a performance in a gradual way, both at the symbolic and signal levels. To this purpose, we discuss a model that applies a smooth morphing among performances with different expressive content, adapting the audio expressive character to the user's desires. Morphing can be realized with a wide range of graduality (from abrupt to very smooth), allowing adaptation of the system to different situations. The sound rendering is obtained by interfacing the expressiveness model with a dedicated postprocessing environment, which allows for the transformation of the event cues. The processing is based on the organized control of basic audio effects. Among the basic effects used, an original method for the spectral processing of audio is introduced.

Nonlinear acoustic systems are often described by means of nonlinear maps acting as instantaneous constraints on the solutions of a system of linear differential equations. This description leads to discrete-time models exhibiting noncomputable loops. We present a solution to this computability problem by means of geometrical transformation of the nonlinearities and algebraic transformation of the time-dependent equations. The proposed solution leads to stable and accurate simulations even at relatively low sampling rates.