Discrete speech representations have gained significant success in a variety of speech-related tasks. Among these, Neural Audio Codec (NAC), which serves as a compressed form of audio signals, have proven effective in speech AIGC applications. Moreover, we believe that the speaker information can be largely preserved in the compression process since the reconstructed voice is almost the same in human listening. In this paper, we explore various training strategies and codec types for NAC-based speaker representation learning. Using ECAPA-TDNN as the model backbone, our approach achieves state-of-the-art performance with a 2.08% EER in NAC-based speaker verification scenarios. To better retain speaker information in early, more compressed layers, we introduce mask-layer augmentation and embedding fusion techniques during the training process. Experimental results show the effectiveness of our methods, particularly when inferring with limited codec layers.

Using larger scale datasets in the training stage of speaker verification model usually leads to better performance. However, when the speaker number of the training dataset becomes extreme large (e.g., more than 1 million), the training speed and GPU memory demand will become bottlenecks which are mainly brought by the extreme large dimension of last fully-connected(FC) layer’s weight matrix. We propose dynamic active sub FC layers (DAS-FC) to tackle this problem. Firstly, all speakers are dynamically divided into speaker groups by clustering rows of last FC layer’s weight matrix. Then, sub FC layers are generated according to speaker groups for model training. We also introduce Mini-Batch K-means and speaker based dataloader to further reduce time and resource costing. Experiments on an extreme large dataset with 1,068,237 speakers show that compared to traditional FC layer, DAS-FC can save up to 87% training time and save 56% GPU memory occupancy with only a 4.2% drop in model performance.

Currently Shenzhen's healthcare institutes and organizations are using disparate IT applications, which are proprietary and home grown at various operation platforms since different years. It is urgent for HB to build up a regional healthcare data center platform so as to exchange and share the information.