In order to study the relationship between emotion and intonation, a new technique is introduced for the extraction of the dominant pitches within speech utterances and the quasi-musical analysis of the multipitch structure. After the distribution of fundamental frequencies over the entire utterance has been obtained, the underlying pitch structure is determined using an unsupervised "cluster" (Gaussian mixtures) algorithm. The technique normally results in 3-6 pitch clusters per utterance that can then be evaluated in terms of their inherent dissonance, harmonic "tension", and "major or minor modality". Stronger dissonance and tension were found in utterances with negative affect, relative to utterances with positive affect. Most importantly, utterances that were evaluated as having positive or negative affect had significantly different modality values. Factor analysis showed that the measures involving multiple pitches were distinct from other acoustical measures, indicating that the pitch substructure is an independent factor contributing to the affective valence of speech prosody.

We report the application of a psychophysical model of pitch perception to the analysis of speech intonation. The model was designed to reproduce the empirical findings on the perception of musical phenomena (the dissonance/consonance of intervals and the tension/sonority of chords), but does not depend on specific musical scales or tuning systems. Application to intonation allows us to calculate the total dissonance and tension among the pitches in the speech utterance. In an experiment using the 144 utterances of 18 male and female subjects, we found greater dissonance and harmonic tension in sentences with negative affect, in comparison with sentences with positive affect.