In this work, we address the problem of novelty detection in the context of image classification, where the goal is to classify whether a query belongs to one of the classes seen during training or to a novel class. Given a network trained on a set of training classes, we utilize the activations from the final fully connected layer in a two-stage framework to determine whether a given query is seen or novel. In the first stage, for a given query, analyzing the top retrieved samples gives us a set of probable categories for that query. For the second stage, a comparator network is designed which can compare features from two samples and classify them into similar or dissimilar pairs. Prototype exemplars from each of the probable categories computed in the first stage are compared with the query feature to determine whether they come from the same or different category. The scores from both the stages are fused using a simple yet effective fuzzy aggregation operator to give a final decision on whether the given query is from a seen or novel class. Extensive experiments on three benchmark databases, MNIST, HASYv2, and Fashion-MNIST demonstrate the effectiveness of the proposed framework.

Recently, near-infrared (NIR) images are increasingly being captured for recognizing faces in low-light/night-time conditions. Matching these images against the controlled high-resolution visible facial images usually present in the database is a challenging task. In surveillance scenarios, the NIR images can have very low resolution and also non-frontal pose which makes the problem even more challenging. In this paper, we propose an orthogonal dictionary alignment approach for addressing this problem. We also propose a re-ranking approach to further improve the recognition performance for each probe by combining the rank list given by the proposed algorithm with that given by another complementary feature/algorithm. Finally, we have also collected our own database Heterogeneous face recognition across Pose and Resolution (HPR) that has facial images captured from two surveillance quality NIR cameras and one high-resolution visible camera, with significant variations in head pose and resolution. Extensive experiments on the modified CASIA NIR-VIS 2.0 database, the Surveillance Camera face database, and our HPR database show the effectiveness of the proposed approaches and the collected database.

Recently, near-infrared (NIR) images are being increasingly used for recognizing facial images across illumination variations and in low-light conditions. In surveillance scenarios, the captured NIR may have low-resolution which results in significant loss of discriminative information along with uncontrolled pose. In this work, we address the challenging task of matching these low-resolution (LR) uncontrolled NIR images with high-resolution (HR) controlled visible (VIS) images usually present in the database. Since the probe and gallery images differ significantly in terms of pose, resolution and spectral properties, we employ a two-stage approach. First, the images are transformed into a common space using metric learning such that the images of the same subject are pushed closer and those of different subjects are pushed apart. We then define an objective function which can simultaneously push both LR NIR and HR VIS samples towards the centroids of the HR VIS samples. We show that the approach is general and can be used for other data like RGB-D and also for matching across pose. Extensive experiments conducted on five datasets shows the effectiveness of our approach.

Cross-domain matching is a challenging problem with several applications like face recognition across pose and resolution, heterogeneous face recognition, etc. Coupled dictionary learning has emerged as a powerful technique for addressing such problems. A novel approach based on aligning two orthogonal dictionaries constructed independently from the two domains is proposed in this work. Once the dictionaries are constructed, the correspondence between the dictionary atoms of the two domains are computed using bipartite graph matching in a common space. A Mahalanobis metric is then derived from sparse coefficient vectors of the aligned dictionaries of the two domains such that the coefficients from data of same class move closer and that of different classes move apart. Unlike other coupled dictionary learning approaches, one-to-one paired training data is not required in the proposed approach. Extensive experiments on MultiPIE, SCFace and MBGC database for face recognition across pose and resolution, CASIA NIRVIS 2.0 database for matching visible to near-infrared face images show the usefulness of the proposed approach for different applications.

We propose a completely automatic approach for recognizing low resolution face images captured in uncontrolled environment. The approach uses multidimensional scaling to learn a common transformation matrix for the entire face which simultaneously transforms the facial features of the low resolution and the high resolution training images such that the distance between them approximates the distance had both the images been captured under the same controlled imaging conditions. Stereo matching cost is used to obtain the similarity of two images in the transformed space. Though this gives very good recognition performance, the time taken for computing the stereo matching cost is significant. To overcome this limitation, we propose a reference-based approach in which each face image is represented by its stereo matching cost from a few reference images. Experimental evaluation on the real world challenging databases and comparison with the state-of-the-art super-resolution, classifier based and cross modal synthesis techniques show the effectiveness of the proposed algorithm.

Pose invariant matching is a very important and challenging problem with various applications like recognizing faces in uncontrolled scenarios, matching objects taken from different view points, etc. In this paper, we propose a discriminative pose-free descriptor (DPFD) which can be used to match faces/objects across pose variations. Training examples at very few representative poses are used to generate virtual intermediate pose subspaces. An image or image region is then represented by a feature set obtained by projecting it on all these subspaces and a discriminative transform is applied on this feature set to make it suitable for classification tasks. Finally, this discriminative feature set is represented by a single feature vector, termed as DPFD. The DPFD of images taken from different viewpoints can be directly compared for matching. Extensive experiments on recognizing faces across pose, pose and resolution on the Multi-PIE and Surveillance Cameras Face datasets and comparisons with state-of-the-art approaches show the effectiveness of the proposed approach. Experiments on matching general objects across viewpoints show the generalizability of the proposed approach beyond faces.