End-to-end Large Speech Language Models (LSLMs) have demonstrated impressive conversational generation abilities, yet consistently fall short of traditional pipeline systems on semantic understanding benchmarks.In this work, we reveal through systematic experimentation that although LSLMs lose some text input performance after speech-text alignment training, the performance gap between speech and text inputs is more pronounced, which we refer to as the modality gap.To understand this gap, we analyze both coarse-and fine-grained text and speech representations.At the coarse-grained level, representations of speech and text in deeper layers are found to be increasingly aligned in direction (cosine similarity), while concurrently diverging in magnitude (Euclidean distance).We further find that representation similarity is strongly correlated with the modality gap.At the fine-grained level, a spontaneous token-level alignment pattern between text and speech representations is observed.Based on this, we introduce the Alignment Path Score to quantify token-level alignment quality, which exhibits stronger correlation with the modality gap.Building on these insights, we design targeted interventions on critical tokens through angle projection and length normalization.These strategies demonstrate the potential to improve correctness for speech inputs.Our study provides the first systematic empirical analysis of the modality gap and alignment mechanisms in LSLMs, offering both theoretical and methodological guidance for future optimization.

Reverberation significantly degrades speech intelligibility, posing a substantial challenge in speech processing. While deep learning advancements offer promising solutions, current methodologies often overlook the effective integration of low-level and high-level feature representations, causing detrimental effects on overall performance. Simultaneously, prior approaches heavily rely on loss functions grounded in quantitative error metrics, which may not fully capture the perceptual intricacies of speech signals. To address these concerns, we introduce CAT-DUnet, a Unet architecture that integrates channel attention, time-frequency attention, and dilated convolution blocks to enhance feature fusion. We innovatively leverage the structural similarity as the training objective to align more closely with human perception, and investigate the effect of applying various reasonable transformations to spectrograms on the performance of the loss function. Through extensive ablation experiments, we demonstrate the effectiveness of our proposed enhancements. Our model outperforms state-of-the-art models on 6 out of 7 metrics, underscoring its exceptional performance.