With the omnipresence of deep learning approaches in modern signal processing, the generalization of models to unseen domains is of crucial importance. Various strategies have been proposed to address such unsupervised domain adaptation including reconstruction autoencoders and adversarial training. In this paper, we propose another strategy using teacher–student learning, where we train a student model for the target domain using pseudo-labels generated by a teacher model trained on the source domain. To improve the adaptation, we propose to exploit cross-version data, i.e., target-domain data without labels but existing in different versions, which are expected to share the same labels. We process the pseudo-labels from the teacher model by keeping only consistent labels or by interpolating conflicting labels. In this paper, we demonstrate the proposed strategy for a music transcription subtask (multi-pitch estimation) using different performances of the same composition. We compare this strategy with a baseline method using a reconstruction autoencoder. Furthermore, we combine our strategy with the baseline method by enforcing the cross-version consistency of the target domain predictions. Our results show that our strategy helps to improve the unsupervised domain adaptation from one instrument (piano) to other instruments (singing and orchestra). Especially for the difficult transfer from piano music to complex orchestral music, we obtain substantial improvements using the proposed approach.

Many tasks in music information retrieval (MIR) involve weakly aligned data, where exact temporal correspondences are unknown. The connectionist temporal classification (CTC) loss is a standard technique to learn feature representations based on weakly aligned training data. However, CTC is limited to discrete-valued target sequences and can be difficult to extend to multi-label problems. In this article, we show how soft dynamic time warping (SoftDTW), a differentiable variant of classical DTW, can be used as an alternative to CTC. Using multi-pitch estimation as an example scenario, we show that SoftDTW yields results on par with a state-of-the-art multi-label extension of CTC. In addition to being more elegant in terms of its algorithmic formulation, SoftDTW naturally extends to real-valued target sequences.

Extracting pitch information from music recordings is a challenging but important problem in music signal processing. Frame-wise transcription or multi-pitch estimation aims for detecting the simultaneous activity of pitches in polyphonic music recordings and has recently seen major improvements thanks to deep-learning techniques, with a variety of proposed model architectures. In this paper, we compare different architectures based on convolutional neural networks, the U-net structure, and self-attention components. We propose several modifications to these architectures including self-attention modules for skip connections, recurrent layers to replace the self-attention, and a multi-task strategy with simultaneous prediction of the degree of polyphony. We compare variants of these architectures in different sizes for multi-pitch estimation, focusing on Western classical music beyond the piano-solo scenario using the MusicNet and Schubert Winterreise datasets. Our experiments indicate that most architectures yield competitive results and that larger model variants seem to be beneficial. However, we find that these results substantially depend on randomization effects and the particular choice of the training–test split, which questions the claim of superiority for particular architectures given only small improvements. We therefore investigate the influence of dataset splits in the presence of several movements of a work cycle (cross-version evaluation) and propose a best-practice evaluation strategy for MusicNet, which weakens the influence of individual test tracks and suppresses overfitting to specific works and recording conditions. A final cross-dataset evaluation suggests that improvements on one specific dataset do not necessarily generalize to other scenarios, thus emphasizing the need for further high-quality multi-pitch datasets in order to reliably measure progress in music transcription tasks.

While global key and chord estimation for both popular and classical music recordings have received a lot of attention, little research has been devoted to estimating the local key for classical music. In this work, we approach local key estimation on a unique cross-version dataset comprising nine performances (versions) of Schubert's song cycle Winterreise-a challenging scenario of high musical ambiguity and subjectivity. We compare an HMM-based system with a CNN-based approach. For both models, we employ a similar training procedure including the optimization of hyperparameters on a validation split. We systematically evaluate the model predictions and provide musical explanations for key confusions. As our main contribution, we explore how different training-test splits affect the models' efficacy. Splitting along the song axis, we find that both methods perform similarly well. Splitting along the version axis leads to clearly higher results, especially for the CNN, which seems to effectively learn the harmonic progressions of the songs ("cover song effect") and successfully generalizes to unseen versions.

While global key and chord estimation for both popular and classical music recordings have received a lot of attention, little research has been devoted to estimating the local key for classical music. Partly, this may be due to its inherent ambiguity and subjectivity, which makes annotating local keys a challenging task. In this article, we approach local key estimation with a cross-version dataset comprising nine performances (versions) of Schubert's song cycle Winterreise annotated by three different music theory experts. We consider two baseline methods that are representative for common types of signal processing algorithms: an HMM-based system and a CNN-based approach. For both models, we employ a similar training procedure including the optimization of hyperparameters on a validation split. We systematically evaluate the model predictions and provide musical explanations for key confusions. As a central contribution, we explore how different training-test splits affect the models' efficacy. Splitting along the song axis, we find that both methods perform similarly well. Splitting along the version axis, we obtain substantially higher accuracies, especially for the CNN, which seems to effectively learn the harmonic progressions of the songs (“cover song effect”) and successfully generalizes to unseen versions. We further discuss the results for several songs in detail and assess our results from the perspective of multiple annotators. This cross-annotator study reveals that a substantial part of the systems' errors coincides with annotator disagreement and an even larger part can be traced back to musically explainable relationships among different keys.

Chords and their progressions are an important aspect of Western tonal music. Specifically, transitions between subsequent chords within a piece carry style-relevant information. To extract such information from audio recordings, a naive approach first peforms automatic chord estimation for computing chord labels explicitly and then derives transition statistics. Often, this is done with Hidden Markov Models involving the Viterbi decoding algorithm. However, since chords are often ambiguous, deciding on one “optimal” chord sequence can be problematic, which heavily affects the subsequent derivation of transition features. In this paper, we propose novel mid-level features that capture chord transitions in a “soft” way. Our method exploits the Baum-Welch algorithm, which does not involve hard decisions on chord labels. Instead, we obtain probabilistic features that account for ambiguities among chords and chord transitions. In several experiments, we evaluate these features within a style classification scenario discriminating four historical periods of Western classical music. Our soft transition features consistently achieve higher accuracies than comparable hard-decision features, thus demonstrating the descriptive power of the novel features.

We propose a set of novel audio features for classifying the style of classical music. The features rely on statistical measures based on a chroma feature representation of the audio data and describe the tonal complexity of the music, independently from the orchestration or timbre of the music. To analyze this property, we use a dataset containing piano and orchestral music from four general historical periods including Baroque, Classical, Romantic, and Modern. By applying dimensionality reduction techniques, we derive visualizations that demonstrate the discriminative power of the features with regard to the music styles. In classification experiments, we evaluate the features' performance using an SVM classifier. We investigate the influence of artist filtering with respect to the individual composers on the classification performance. In all experiments, we compare the results to the performance of standard features. We show that the introduced features capture meaningful properties of musical style and are robust to timbral variations.