Creating synthetic behaviors of vehicles in simulation applications has always been challenging from a development standpoint. First, it is a real challenge to create a credible and realistic simulation while achieving the required runtime efficiency. Second, the effort required to implement it can add significant cost to the development processes. In this paper, we propose an automated way to design vehicle simulation systems by transfer learning from reality to simulators. Our methods rely on advanced deep learning technologies and datasets commonly used in the field of self-driving cars. To assess how well this approach would work in a simulation environment, experiments using the CARLA simulator are presented in the evaluation. The results show that the proposed transfer learning approach provides good results, both quantitatively and qualitatively, and is suitable for runtime evaluation even in resource-constrained simulation applications such as video games.