In this paper, we use motion and appearance contexts for persistent tracking of objects in aerial imagery. The motion context in a given environment is a collection of trajectories of objects which are representative of the motion of the occluded or unobserved object. It is learned using a clustering scheme based on the Lyapunov characteristic exponent (LCE) which measures the mean exponential rate of divergence of the nearby trajectories. The learned motion context is then used in a regression framework to predict the location of the unobserved object. The appearance context of an occluded (target) object consists of appearance information of objects which are currently occluded or unobserved. It is incorporated by learning a distribution of interclass variation for each target-unobservable object pair. In addition, intra-class variation distribution is constructed for each occluded object using all of its previous observations. Qualitative and quantitative results are reported on challenging aerial sequences.