Abstract Class imbalance is a crucial challenge in classification tasks, and in recent years, with the advancements in deep learning, research on oversampling techniques based on GANs has proliferated. These techniques have proven to be excellent in addressing the class imbalance issue by capturing the distributional features of minority samples during training and generating high-quality new samples. However, oversampling methods based on GANs may suffer from gradient vanishing, resulting in mode collapse, and produce noise and boundary-blurring issues when generating new samples. This paper proposes a novel oversampling method based on a conditional GAN (CGAN) incorporating Wasserstein distance. It generates an initial balanced dataset from minority class samples using the CGAN oversampling approach and then uses a noise and boundary recognition method based on K-means and $$k$$ k nearest neighbors algorithm to address the noise and boundary-blurring issues. The proposed method generates new samples that are highly consistent with the original sample distribution and effectively solves the problems of noise data and class boundary blurring. Experimental results on multiple public datasets show that the proposed method achieves significant improvements in evaluation metrics such as Recall, F1_score, G-mean, and AUC.

At a time when end-to-end speech recognition technology is becoming more and more popular, we conduct research on various end-to-end speech technologies, and use the Transformer-based speech framework to study and find that its multi-head attention is not effective in local feature acquisition. And in the face of noise problems in real scenes, the training convergence speed is too slow. In order to solve the problems caused by Transformer, a new speech recognition framework based on MCNN-Transformer-CTC speech recognition method is proposed. Through MCNN (multi-stream convolutional neural network) in the pre-acoustic unit through multiple parallel channels Local feature extraction is carried out in terms of time width and spectral capability, which makes up for the lack of self-attention mechanism in local feature extraction, and the multitask learning method is used to add CTC structure to make up for the problem of slow training convergence. The training effect of this model on the Aishell1 dataset has reached a CER of 6.23%, which is a further improvement compared to the Transformer model.