We introduce a novel machine learning-based fusion model termed as PI-LSTM (Physics-Infused Long Short-Term Memory Networks) that integrates first principle Physics-Based Models and Long Short-Term Memory (LSTM) network. Our architecture aims at combining equation-based models with data-driven machine learning models to enable accurate predictions of complex dynamic systems. In this hybrid architecture, recurrency aids the temporal memory of the inputs and output of the partial physics model, in a way that facilitates generalization with scarce data sets. We illustrate the application of PI-LSTM on two dynamical systems namely Inverted Pendulum and Tumor Growth. Empirical results on both test problems stand witness to the effectiveness of using physics in guiding machine learning models and the superiority of the outlined hybrid model over purely data-driven models.