The Moon’s distinctive spatial weathering environment and the internal dynamics of its evolutionary process have produced intricate lunar characteristics that make remote sensing data interpretation challenging. The variety of features and the absence of observational data have resulted in the issue of ill-posed inversion in lunar remote sensing. The Hapke radiative transfer model employed in the current inversion is often oversimplified by idealized assumptions, including applying a single-parameter Gaussian distribution to represent macroscopic roughness, significantly diverging from the actual lunar surface characteristics. In addition, the macroscopic roughness parameter of the Hapke model has some coupling with the single-scattering albedo, making inversion difficult. This study proposes an enhanced Hapke model defined as RaySHapke, to accurately represent the actual lunar surface parameters and simulate shading, shadowing, and multiple scattering effects by raytracing. This approach presents the macroroughness shadowing function and proposes an approximate Bayesian inversion framework to enhance the stability and efficiency of the inversion process. The results indicate that raytracing can proficiently replicate the terrain effect. Comparison of the original Hapke model with the RaySHapke model suggests that the raytracing shade function may enhance the accuracy and stability of the inversion of single-scattering albedo and phase function parameters. This study provides a solid foundation for interpreting the physical properties of the lunar surface, as well as lunar science and engineering analysis.