Recently self-supervised learning has emerged as an effective approach to improve the performance of automatic speech recognition (ASR).Under such a framework, the neural network is usually pre-trained with massive unlabeled data and then fine-tuned with limited labeled data.However, the nonstreaming architecture like bidirectional transformer is usually adopted by the neural network to achieve competitive results, which can not be used in streaming scenarios.In this paper, we mainly focus on improving the performance of streaming transformer under the self-supervised learning framework.Specifically, we propose a novel two-stage training method during finetuning, which combines knowledge distilling and self-training.The proposed training method achieves 16.3% relative word error rate (WER) reduction on Librispeech noisy test set.Finally, by only using the 100h clean subset of Librispeech as the labeled data and the rest (860h) as the unlabeled data, our streaming transformer based model obtains competitive WERs 3.5/8.7 on Librispeech clean/noisy test sets.

Cover song identification is a challenging task in the field of Music Information Retrieval (MIR) due to complex musical variations between query tracks and cover versions. Previous works typically utilize hand-crafted features and alignment algorithms for the task. More recently, further breakthroughs are achieved by employing neural network approaches. In this paper, we propose a novel Convolutional Neural Network (CNN) towards cover song identification. We train the network through classification criteria. Having been trained, the network is used to extract music representation for cover song identification. A training scheme is designed to train robust models against tempo changes. Experimental results show that our approach outperforms state-of-the-art methods on several public datasets with low time complexity.

Cover song identification is an important problem in the field of Music Information Retrieval. Most existing methods rely on hand-crafted features and sequence alignment methods, and further breakthrough is hard to achieve. In this paper, Convolutional Neural Networks (CNNs) are used for representation learning toward this task. We show that they could be naturally adapted to deal with key transposition in cover songs. Additionally, Temporal Pyramid Pooling is utilized to extract information on different scales and transform songs with different lengths into fixed-dimensional representations. Furthermore, a training scheme is designed to enhance the robustness of our model. Extensive experiments demonstrate that combined with these techniques, our approach is robust against musical variations existing in cover songs and outperforms state-of-the-art methods on several datasets with low time complexity.

So far, few cover song identification systems that utilize index techniques achieve great success. In this paper, we propose a novel approach based on skipping bigrams that could be used for effective index. By applying Vector Quantization, our algorithm encodes signals into code sequences. Then, the bigram histograms of code sequences are used to represent the original recordings and measure their similarities. Through Vector Quantization and skipping bigrams, our model shows great robustness against speed and structure variations in cover songs. Experimental results demonstrate that our model achieves better performance than recent methods and is less computationally demanding.

Cover song identification has long been a challenging task due to key, timbre and structure variations in different renditions of a song. Previous research mostly involves handcrafted features and sequence alignment methods, where further breakthroughs can hardly be achieved. In this paper, we utilize a supervised deep learning method to learn an effective feature extractor for cover song identification. Regarding it as a classification problem, we propose a key-invariant convolutional neural network robust against key transposition for classification. Having been trained, the network is used to extract representations of music, which could be used to measure the similarity between songs. Besides, the representations are highly sparse; effective algorithms can be devised to accelerate the computation. Experimental results show that our model achieves high precision with low time cost on several datasets. Especially, our method outperforms the state-of-the-art approach on one of the datasets.

Chroma is a widespread feature for cover song recognition, as it is robust against non-tonal components and independent of timbre and specific instruments. However, Chroma is derived from spectrogram, thus it provides a coarse approximation representation of musical score. In this paper, we proposed a similar but more effective feature Note Class Profile (NCP) derived with music transcription techniques. NCP is a multi-dimensional time serie, each column of which denotes the energy distribution of 12 note classes. Experimental results on benchmark datasets demonstrated its superior performance over existing music features. In addition, NCP feature can be enhanced further with the development of music transcription techniques. The source code can be found in github1.