This paper addresses the passive source localization problem using hybrid angle-of-arrival (AOA) and time-difference-of-arrival (TDOA) measurements observed by single stationary receiver at several time intervals, when the source is moving with a constant velocity trajectory. We first show that the localization problem cannot be achieved using a sequence of one kind of measurements (i.e., AOAs only or TDOAs only), but can be accomplished by using both AOA and TDOA observations together. Then we develop a parametric model by integrating AOA and TDOA, and then formulate a non-convex constrained weighted least squares (CWLS) problem that is applicable to the 2-D and 3-D scenarios for locating the source. Benefiting from the integrated model, the global solution of the CWLS problem is guaranteed without having local convergence or divergence issues. Furthermore, we conduct the mean square error analysis to validate that the proposed method is able to reach the Cramer-Rao lower bound accuracy under small Gaussian noise for the 2-D and 3-D cases. Simulation results also confirm the good performance of the proposed method.