Personalized speech enhancement (PSE) methods typically rely on pre-trained speaker verification models or self-designed speaker encoders to extract target speaker clues, guiding the PSE model in isolating the desired speech. However, these approaches suffer from significant model complexity and often underutilize enrollment speaker information, limiting the potential performance of the PSE model. To address these limitations, we propose a novel Speaker Encoder-Free PSE network, termed SEF-PNet, which fully exploits the information present in both the enrollment speech and noisy mixtures. SEF-PNet incorporates two key innovations: Interactive Speaker Adaptation (ISA) and Local-Global Context Aggregation (LCA). ISA dynamically modulates the interactions between enrollment and noisy signals to enhance the speaker adaptation, while LCA employs advanced channel attention within the PSE encoder to effectively integrate local and global contextual information, thus improving feature learning. Experiments on the Libri2Mix dataset demonstrate that SEF-PNet significantly outperforms baseline models, achieving state-of-the-art PSE performance. Our source code is available at https://github.com/isHuangZiling/SEF-PNet.

When there’s a mismatch between the training and test domains, supervised speech enhancement (SE) models trained on synthetic paired noisy-clean data often struggle in real-world scenarios, highlighting the industry’s strong demand for unsupervised training and domain adaptation methods. In this study, we introduce PHA-ReMixIT, a novel approach for leveraging out-of-domain (OOD) noise signals to enhance unsupervised domain adaptation in SE. Our method builds upon the state-of-the-art ReMixIT by introducing a paired unsupervised remixing technique, which augments the diversity of target domain training data with OOD noise signals. We further propose a heterogeneous noise invariant training to align the OOD augmented noisy mixtures with their paired heterogeneous counterparts, encouraging the model to output cleaner speech. Additionally, an adaptive focal weighting mechanism is also introduced to dynamically emphasize the data importance of both in-domain and OOD noisy mixtures during model adaptation. Experiments on CHiME-7 unsupervised domain adaptation for conversational speech enhancement (UDASE) task demonstrate that PHA-ReMixIT significantly outperforms the ReMixIT baseline, boosting SE performance on both real and synthesized test sets.