Multi-source remote sensing data classification refers to the process of categorizing ground objects by integrating complementary strengths of multiple remote sensing data, such as hyperspectral image (HSI), light detection and ranging (LiDAR) and synthetic aperture radar (SAR) data. However, current Mamba-based multisource remote sensing data classification approaches rely on fixed scanning patterns that are inadequate in characterizing spectral-spatial information. Additionally, current fusion techniques adopt concatenation or attention-based fusion rules without considering the complementary characteristics between different modalities. To address these limitations, we propose a spectral-spatial dynamic scan Mamba (SDSM) for multi-source remote sensing data classification. Specifically, a dynamic scan Mamba network is proposed to extract the spectral-spatial features of multi-source remote sensing data, in which a dynamic scan module is designed to adaptively capture the important spatial and spectral information. Furthermore, a bidirectional cross-modal fusion rule is proposed to merge the extracted features, in which a global-local frequency feature extraction module is designed to extract the salient structural features of multi-source remote sensing data as clues to guide heterogeneous feature fusion. Comprehensive experiments on four multi-source remote sensing datasets, i.e., MUUFL, Augsburg, Italy and Yellow River, demonstrate that the proposed method outperforms other state-of-the-art methods with respect to quantitative and qualitative results. The code of this article is available at https://github.com/PuhongDuan/SDSM.