Human face recognition is one of the most important research areas in biometrics. However, the robust face recognition under a drastic change of the facial pose, expression, and illumination is a big challenging problem for its practical application. Such variations make face recognition more difficult. In this paper, we propose a novel face recognition method, called Attentional Feature-pair Relation Network (AFRN), which represents the face by the relevant pairs of local appearance block features with their attention scores. The AFRN represents the face by all possible pairs of the 9x9 local appearance block features, the importance of each pair is considered by the attention map that is obtained from the low-rank bilinear pooling, and each pair is weighted by its corresponding attention score. To increase the accuracy, we select top-K pairs of local appearance block features as relevant facial information and drop the remaining irrelevant. The weighted top-K pairs are propagated to extract the joint feature-pair relation by using bilinear attention network. In experiments, we show the effectiveness of the proposed AFRN and achieve the outstanding performance in the 1:1 face verification and 1:N face identification tasks compared to existing state-of-the-art methods on the challenging LFW, YTF, CALFW, CPLFW, CFP, AgeDB, IJB-A, IJB-B, and IJB-C datasets.

An image pyramid can extend many object detection algorithms to solve detection on multiple scales. However, interpolation during the resampling process of an image pyramid causes gradient variation, which is the difference of the gradients between the original image and the scaled images. Our key insight is that the increased variance of gradients makes the classifiers have difficulty in correctly assigning categories. We prove the existence of the gradient variation by formulating the ratio of gradient expectations between an original image and scaled images, then propose a simple and novel gradient normalization method to eliminate the effect of this variation. The proposed normalization method reduce the variance in an image pyramid and allow the classifier to focus on a smaller coverage. We show the improvement in three different visual recognition problems: pedestrian detection, pose estimation, and object detection. The method is generally applicable to many vision algorithms based on an image pyramid with gradients.

Recently, deep convolutional neural networks have set a new trend in fields of face recognition by improving the state-of-the-art performance. By using deep neural networks, much more sophisticated and high level abstracted features can be learned automatically. In this paper, we propose a method for face recognition using multi-scale convolution layer blocks and triplets of faces in unconstrained environments. We use the ensemble of deep convolution neural networks trained on differently scaled and aligned face images. This extracts low dimensional but high-level abstraction and discriminative features for face recognition. With these features, we employ the jointly Bayesian model and transfer learning which adapts the knowledge trained from the source domain to target domain. Experiment shows that our proposed method achieves 98.33% pair-wise verification accuracy on the LFW dataset.

Learning valid similarities is a vital problem in hashing methods, especially in large-scale image search. Similarity pertained hashing method is widely used in image retrieval for its high quality compact binary code mapping. The hashing scheme of most existing hashing methods is that the input data is encoded as a vector of visual features and hashed into binary hash codes via projection functions or quantization methods afterward. However, this separated pipeline may prone to lose accurate similarities of images, since the limited compatible domain between visual feature vectors generation and binary codes mapping process. Encouraged by the extraordinary image representation learning ability of deep neural networks in classification, we propose a structure that merges binary code generation process within deep neural networks for efficient image retrieval. The proposed architecture contains two fundamental blocks. The stacked convolution layers of Network In Network with global average pooling compute effective image representation and the embedded latent layer with binary activation functions learn binary hash codes simultaneously. Experiments show that the proposed method gains improvement over several state-of-the-art hashing methods.

Recently, thanks to a variety of sensors equipped on smartphone, a lot of research about mobile activity recognition using accelerometer have been studied for context inference of mobile user and healthcare applications. Previous works, however, have a limitation in classifying some activities because of intra-class variations and inter-class similarities. To handle this problem, in this paper we propose a novel method to recognize activity of smart phone user based on hidden Markov model, where an ensemble method of hidden Markov models is proposed and used to recognize activity. To evaluate our method, we have carried out some experiments by using UCI Human Activity Recognition dataset, and as a result we have achieved about 83.51% accuracy when using two simple features, mean and standard deviation. It is a comparable result to other powerful discriminative methods such as support vector machine and multilayer perceptron.

Abandoned object and removed object are important abnormal objects in visual surveillance area to predict the crimes such as explosion or theft event. In real situations, most of existing methods using CCD camera show inconsistent performance because they use a lot of threshold values depending on the environmental conditions of target scene such as illumination change, high traffic volume and complex background. We propose a nonparametric state machine with hierarchical structure consisting of three layers. As shown in the experimental results, the proposed method can be applied to general situations because the state transitions is performed by trained SVM classifiers.

Abnormal object detection and discrimisnation is a critical research area for vision-based surveillance systems. This paper proposes a novel algorithm for the detection and discrimination of abnormal objects, such as abandoned and stolen objects. The proposed algorithm consists of three stages and three different filters. The three stages cooperate with each other using the feedforward model to enhance detection and discrimination performance, while the sequential filters efficiently reject falsely detected regions using three categories of information. The results of experiments conducted using public datasets indicate that the proposed algorithm is more accurate and has a lower false alarm ratio than the existing system.

We propose the efficient and accurate hierarchical ICIA fitting method for 3D Morphable Models (3DMMs). The conventional ICIA fitting method for 3DMMs requires a long computation time because the 3D face model contains a large number of vertices and it also requires to compute the Hessian matrix using the visible vertices every iteration. For the efficient fitting, we use the hierarchical fitting that use a set of multi-resolution 3D face model and the Gaussian image pyramid. For more accurate fitting, we use a two-stage parameter update that only update the rigid and the texture parameters and then update all parameters after the initial convergence. We present several experiment results to prove that our proposed method shows better performance than previous works.

We constructed a face database POSTECH face database (PF07). PF07 contains the true-color face images of 200 people, 100 men and 100 women, representing 320 various images (5 pose variations times 4 expression variations times 16 illumination variations) per person. All of the people in the database are Korean. We also present the results of face recognition experiments under various conditions using three baseline face recognition algorithms in order to provide an example evaluation protocol on the database. The database is expected to be used to evaluate the algorithm of face recognition for Korean people or for people with systematic variations.

The inverse compositional image alignment (ICIA) is known as an efficient matching method for 3D morphable models (3DMMs). However, it requires a long computation time since the 3D face models consist of a large number of vertices. Also, it requires to recompute the Hessian matrix using the visible vertices every iteration. For a fast and an efficient matching, we propose the efficient and accurate hierarchical ICIA (HICIA) matching method for 3DMMs. The proposed matching method requires multi-resolution 3D face models and the Gaussian image pyramid. The multi-resolution 3D face models are built by sub-sampling at the 2:1 sampling rate to construct the lower-resolution 3D face models. For more accurate matching, we use a two-stage model parameter update that only updates the rigid and the texture parameters and then updates all parameters after the initial convergence. We present several experimental results to prove that the proposed method shows better performance than that of the conventional ICIA matching method.