MOTIVATION: RNA velocity has become a powerful tool for uncovering transcriptional dynamics in snapshot single-cell data. However, current RNA velocity approaches often assume constant transcriptional rates and treat genes independently with gene-specific times, which may introduce biases and deviate from biological realities. Here, we present InterVelo, a novel deep learning framework that simultaneously learns cellular pseudotime and RNA velocity. RESULTS: InterVelo leverages an unsupervised cellular time to guide RNA velocity estimation, while the estimated RNA velocity in turn refines the direction of pseudotime. By benchmarking InterVelo against existing methods on both simulated and real datasets, we demonstrate its superior performance in recovering pseudotime and RNA velocity. InterVelo yields more precise velocity estimations in terms of both direction and magnitude, with outstanding robustness across diverse scenarios. Furthermore, it successfully identifies driver genes and enables reliable gene activity enrichment analysis. The flexible architecture of InterVelo also allows for the integration of multi-omic data, enhancing its applicability to complex biological systems. AVAILABILITY AND IMPLEMENTATION: InterVelo is implemented using Python, and the code is available on GitHub https://github.com/yurouwang-rosie/InterVelo and has been archived with a DOI https://doi.org/10.5281/zenodo.16158798 for reproducibility.