Automatic Music Transcription (AMT) aims to convert music audio into digital sheet music. Piano transcription is a popular but challenging subtask of AMT. For every piano pitch, the harmonic structure is fixed in the frequency domain, while the Transformer based on self-attention has great potential to extract features in the long sequence. In this paper, we propose piano harmonic attention, a mask self-attention, for better capturing harmonic features. The mask matrix is designed with the harmonic prior to pre-modeling the harmonic structure during calculating attention scores. To verify its effectiveness, we append the harmonic attention-based Transformer after every convolutional neural network block of the High-resolution piano transcription system. The evaluation results on the MAESTRO dataset show that the proposed model achieves comprehensive improvements over the baseline, with a note F1 score of 97.33%, which is comparable to the state-of-the-art system.

Piano transcription is a significant problem in music information retrieval, which aims to infer the note sequence from recorded music signals. Recently, visual piano transcription heavily relies on Convolutional Neural Networks (CNNs) for local feature detection, facing challenges in capturing global features of the piano keyboard. Therefore, we propose a novel visual piano transcription model that integrates CNN and transformer branches. The CNN branch has inductive bias and extracts local features of the piano key, while the transformer branch aggregates global features over the keyboard. The proposed model integrates local features and global features at different resolutions, enhancing the performance of the visual transcription model. Finally, our model achieves a remarkable F1-score of 92.35% on the OMAPS2 dataset and attains state-of-the-art results on other datasets. This substantiates the model’s innovative approach and its potential to advance visual piano transcription within music information retrieval.

Piano transcription is a significant problem in the field of music information retrieval, aiming to obtain symbolic representations of music from captured audio or visual signals. Previous research has mainly focused on single-modal transcription methods using either audio or visual information, yet there is a small number of studies based on audio-visual fusion. To leverage the complementary advantages of both modalities and achieve higher transcription accuracy, we propose a two-stage audio-visual fusion piano transcription model based on the attention mechanism, utilizing both audio and visual information from the piano performance. In the first stage, we propose an audio model and a visual model. The audio model utilizes frequency domain sparse attention to capture harmonic relationships in the frequency domain, while the visual model includes both CNN and Transformer branches to merge local and global features at different resolutions. In the second stage, we employ cross-attention to learn the correlations between different modalities and the temporal relationships of the sequences. Experimental results on the OMAPS2 dataset show that our model achieves an F1-score of 98.60%, demonstrating significant improvement compared with the single-modal transcription models.