Inverse synthetic aperture ladar (ISAL) can perform high-resolution imaging for satellites. However, due to the short wavelength of the laser, satellite micro-vibration will introduce space-variant vibration phase error (SVVPE) and space-invariant vibration phase error (SIVVPE) in the echoes, which seriously blur the ISAL image. In this paper, we propose a noise-robust vibration phase compensation algorithm to accurately estimate and correct these two types of vibration phase errors by frequency descent minimum entropy optimization. Firstly, considering the characteristics of the micro-vibration of satellites, we establish a novel phase error model based on the Fourier series theory, which only contains low-frequency vibration components. The estimation of the phase errors is then translated into the estimation of the model’s Fourier coefficients, which can be achieved by a multi-dimensional minimum entropy optimization. After that, a frequency descent method (FD) is proposed to transform the multi-dimensional optimization into a group of two-dimensional optimizations so that the proposed algorithm can achieve monotonic iterative convergence. In addition, we introduce a solution space adaptive reduction operation to reduce the computational burden when solving the two-dimensional minimum entropy optimizations by the genetic algorithm (GA) to obtain the global optimal solution. Finally, experiments based on the simulated data and the real measured data confirm the effectiveness of the proposed algorithm. Compared with the traditional methods, the proposed algorithm achieves higher phase error estimation accuracy and better image quality.