Recently, recognizing spontaneous facial expression has gained increasing attention in various emerging applications related to human affect. Spontaneous facial expression may generally have different temporal characteristics across subjects, emotion types, and so on. In this paper, we proposed a facial expression recognition (FER) method which adaptively magnifies a subtle facial motion based on its temporal characteristics. In training stage, we learn the relations between the temporal characteristics of facial motions and their discriminative temporal filtering. The learned model is used to automatically predict the most discriminative temporal filtering that magnifies the subtle facial motion in a test sequence. Experimental result shows that the proposed FER using the adaptive motion magnification performed clearly better than FER using non-adaptive motion magnification as well as FER without motion magnification.