A mobility event occurs when a passenger moves out or takes off from a particular location. Mobility event prediction is of utmost importance in the field of intelligent transportation systems. It has a huge potential in solving important problems such as minimizing passenger waiting time and maximizing the utilization of the transportation resources by planning vehicle routes and dispatching transportation resources. Recently, numerous mobility pattern mining methods have been proposed to predict the transportation supply and demand in different locations. Those methods first reveal the event patterns of each Place of Interests (POI) independently and then employ a separate region function as a post-processing step. This separate process, that disregards the intrinsic spatial and temporal pattern correlations between POI, is sub-optimal and complex, resulting in a poor generalization in different scenarios. In this work, we propose a Spatial-Temporal mobility Event Prediction framework based on Deep neural network (StepDeep) for simultaneously taking into account all correlated spatial and temporal mobility patterns. StepDeep not only simplifies the prediction process but also enhances the prediction accuracy. Our StepDeep proposes a novel problem formulation towards an end-to-end mobility prediction framework, that is, switching mobility events over time in an area into an event video and then posing the mobility prediction problem as a video prediction task. Such a novel formulation can naturally encode spatial and temporal dependencies for each POI. StepDeep thus predicts the spatial-temporal events by incorporating the new time sensitive convolution filters, spatial sensitive convolution filters, and spatial-temporal sensitive convolution filters into a single network. We conduct experimental evaluations on a real-world 547-day New York City taxi trajectory dataset, which show that StepDeep provides higher prediction accuracy than five existing baselines. Moreover, StepDeep is generalizable and can be applied to numerous spatial-temporal event prediction scenarios.