Despite the development of computer vision techniques, the micro-expression (ME) recognition task still remains a great challenge because MEs have very low intensity and short duration. However, the ME recognition is of great significance since it provides important clues for real affective states detection. This paper proposes a novel Block Division Convolutional Network (BDCNN) with the implicit deep features augmentation. In detail, BDCNN learns from four optical flow features computed by the onset and apex frames of each video. It innovatively divides each image into a set of small blocks in the deep learning model, then the convolution and pooling operations are performed on these small blocks in sequence. To handle the small sample size problem in the micro-expression data, this study uses the improved implicit semantic data augmentation algorithm in the deep features space. Experiments are conducted on three publicly available databases, viz, CASME II, SMIC, and SAMM. Experimental results show that our model outperforms the state-of-the-art methods by attaining the accuracy of 84.32% and F1-score of 82.13% on the 3-class datasets, and the accuracy of 81.82% and F1-score of 75.46% on the 5-class datasets, respectively. Our source code is publicly available for non-commercial or research use athttps://github.com/MLDMXM2017/BDCNN.