The Mixture of Experts (MoE) model is a promising approach for handling code-switching speech recognition (CS-ASR) tasks. However, the existing CS-ASR work on MoE has yet to leverage the advantages of MoE’s parameter scaling ability fully. This work proposes DLG-MoE, a Dynamic Language Group-based MoE, which can effectively handle the CS-ASR task and leverage the advantages of parameter scaling. DLG-MoE operates based on a hierarchical routing mechanism. First, the language router explicitly models the language attribute and dispatches the representations to the corresponding language expert groups. Subsequently, the unsupervised router within each language group implicitly models attributes beyond language and coordinates expert routing and collaboration. DLG-MoE outperforms the existing MoE methods on CS-ASR tasks while demonstrating great flexibility. It supports different top-k inference and streaming capabilities and can also prune the model parameters flexibly to obtain a monolingual sub-model.

Recently, flow matching based speech synthesis has significantly enhanced the quality of synthesized speech while reducing the number of inference steps. In this paper, we introduce SlimSpeech, a lightweight and efficient speech synthesis system based on rectified flow. We have built upon the existing speech synthesis method utilizing the rectified flow model, modifying its structure to reduce parameters and serve as a teacher model. By refining the reflow operation, we directly derive a smaller model with a more straight sampling trajectory from the larger model, while utilizing distillation techniques to further enhance the model performance. Experimental results demonstrate that our proposed method, with significantly reduced model parameters, achieves comparable performance to larger models through one-step sampling.

Noisy labels are inevitable in real-world datasets. These noisy labels cause deep neural networks to gradient descent towards the wrong direction, leading to performance degradation. In this paper, We propose an efficient method for filtering out noisy labels during training. We calculate an embedding center for each speaker and compute the cosine similarity between each audio embedding and its corresponding speaker embedding center. Subsequently, we use a Gaussian Mixture Model (GMM) to classify the samples into two categories based on their cosine similarities and filter out the class with lower similarity as noisy labels. These steps are iterated until training is complete. We conducted experiments on real-world datasets as well as scenarios with artificially added noisy labels of different types. The experimental results demonstrate that our method achieved notable performance.