Cross-project defect prediction (CPDP) aims to identify defect-prone modules in a target project by leveraging data from external source projects. Over time, research has shifted from single-source to multi-source CPDP to increase data diversity and capture broader defect patterns. However, naively combining multiple sources can introduce substantial data divergence and degrade performance. A key limitation is that many methods overlook intra-project heterogeneity, treating each source project as uniform. Moreover, data selection typically relies on independent metric matching, which ignores structural relationships among software metrics. To address these limitations, we propose adaptive spectral clustering and structural alignment (ASCSA), a unified framework that combines adaptive spectral clustering and structural alignment. First, adaptive spectral clustering partitions each source project into coherent clusters, mitigating intraproject heterogeneity. Second, a structural alignment method selects source clusters that preserve higher-order metric relationships with the target, avoiding misleading matches based solely on marginal distributions. Finally, a deep neural network with maximum mean discrepancy (MMD) loss minimizes residual distribution gaps to enable effective knowledge transfer. In this study, we conduct comprehensive experiments on 25 widely used software projects. The results show that ASCSA consistently outperforms state-of-the-art methods, achieving improvements in AUC ranging from 5.26% to 23.08%, and in MCC from 15.62% to 68.18%. These findings highlight the effectiveness of jointly addressing intra-project heterogeneity, and structural alignment for reliable cross-project defect prediction.