In this paper we describe a novel approach to applying text-based information retrieval techniques to music collections. We represent tracks with a joint vocabulary consisting of both conventional words, drawn from social tags, and audiomuswords, representing characteristics of automatically-identified regions of interest within the signal. We build vector space and latent aspect models indexing words and muswords for a collection of tracks, and show experimentally that retrieval with these models is extremely well-behaved. We find in particular that retrieval performance remains good for tracks by artists unseen by our models in training, and even if tags for their tracks are extremely sparse.

We describe a method of segmenting musical audio into structural sections based on a hierarchical labeling of spectral features. Frames of audio are first labeled as belonging to one of a number of discrete states using a hidden Markov model trained on the features. Histograms of neighboring frames are then clustered into segment-types representing distinct distributions of states, using a clustering algorithm in which temporal continuity is expressed as a set of constraints modeled by a hidden Markov random field. We give experimental results which show that in many cases the resulting segmentations correspond well to conventional notions of musical form. We show further how the constrained clustering approach can easily be extended to include prior musical knowledge, input from other machine approaches, or semi-supervision.

Four music segmentation algorithms are presented, one based on purely timbral features, one on purely harmonic features, and two on different combinations of features. They are compared against each other and against human annotations of two albums by The Beatles. Example segmentations are given together with a quantitative measure of boundary accuracy. No algorithm is found to be clearly superior, although examples suggest that the combined algorithms can offer improved boundary detection.

A method for segmenting musical audio with a hierarchical timbre model is introduced. New evidence is presented to show that music segmentation can be recast as clustering of timbre features, and a new clustering algorithm is described. A prototype thumbnail-generating application is described and evaluated. Experimental results are given, including comparison of machine and human segmentations