Various recent advancements in deep learning models have greatly boosted the performance of semantic pattern recognition using images. Various state estimation of an individual like emotional state and other certain character features or traits can be estimated from the facial images. With this motivation, in this work we are attempting to infer criminal tendency or (crime prediction/detection) from facial images by using the learning capabilities of various deep learning architectures. More precisely two type of deep learning models we have used in this study: standard convolutional neural network(CNN) architecture and pre-trained CNN architectures, namely VGG-16, VGG-19, and Incep-tionV3. We have done a performance comparative analysis among these models for efficiently capturing criminal traits from a human face. The efficacy of the above deep learning models was evaluated on a public database, National Institute of Standards and Technology (NIST). To avoid any discrepancies, we have only used male images in this work. It was found that VGG CNN models are best performing models, especially in a limited data scenario producing the classification accuracy of 99.5% in identifying criminal faces.