Fingerprint Presentation Attack Detection (PAD) is crucial for ensuring the reliability of fingerprint recognition systems. Mainstream methods typically frame PAD as a binary classification task. Although they achieve commendable results, these approaches often fail on unseen spoofs. This is because their decision boundaries rely heavily on trainset spoofs, leading to poor generalization. To address this issue, we reformulate PAD as an anomaly detection problem and propose the Fixed-Prototype PAD method. First, we introduce Fixed-Prototype Contrastive Learning (FPCL) for extracting PAD features. FPCL clusters the features of live fingerprints near a fixed prototype while pushing the features of spoof fingerprints away. We then design a Fixed-Prototype Anomaly Detection method to replace binary classification for decision-making. Specifically, this involves comparing the distance between test fingerprint features and the fixed live prototype to detect spoofs. Our method eliminates the dependency on the trainset spoofs during decision-making, thereby enhancing generalization capability. Experimental results demonstrate that our approach achieves state-of-the-art performance on the LivDet 2019 and LivDet 2021 benchmarks.

Fingerprint Presentation Attack Detection (PAD) is a crucial step in automatic fingerprint identification systems, which safeguards users from unauthorized malicious access. However, current presentation attack (i.e. spoof) techniques can forge intricate details of fingerprints (such as sweat holes), which makes the artifact evidence harder to detect. In this paper, we propose a novel PAD method from the perspective of decomposition to highlight the artifact evidence in each constituent element. Specifically, we utilize the fingerprint enhancement to decompose the fingerprint into the ridge region and the edge region. We observe that artifact evidence mainly exists in the gradient field within the ridge region, while it primarily resides in the spatial domain within the edge region. Then we propose an Orientation-Based Central Difference Convolution (OB-CDC) layer to prioritize gradient variations along the ridge direction. To further enhance robustness, we propose a Minutia Patches Random Rotation (MPRR) operation to disrupt the identity information of the fingerprint while preserving the artifact evidence. By integrating these techniques, we propose a two-stream network called Presentation-Attack-Detection-with-Region-Decomposition-Network (PADRD-Net) which integrates the processed feature of the ridge region and the edge region through a halfway fusion ResNet-18 structure. Experimental results on the LivDet 2021 dataset show that our proposed PADRD-Net can achieve 20.39% on BPCER@APCER = 1% and 87.12% on TDR@FDR = 1%, significantly outperforms the state-of-the-art. We also achieve outstanding performance in both the cross-sensor scenario and the cross-sensor and cross-material scenario. Extensive ablation studies and analysis experiments further indicate the effectiveness and robustness of our method.

Fingerprint recognition is a crucial biometric technology extensively used in identity verification, including areas like criminal investigations, security systems, and biometric authentication. This technology encounters greater challenges when dealing with incomplete fingerprint images, especially those with significant background noise or substantial portions of the fingerprint missing. Existing incomplete fingerprint recognition technologies struggle with extensive data loss, primarily due to the significant reduction and difficulty in extracting usable features from incomplete fingerprint images. Current image processing methods or deep learning models are unable to comprehensively reconstruct fingerprint features with limited information. To address these challenges, we introduce the Finger Recovery Transformer (FingerRT), an innovative network specifically designed for recovering incomplete fingerprint information. FingerRT can simultaneously complete ambient noise cancellation and fingerprint feature information recovery, resulting in a complete and clean fingerprint image. FingerRT combines the most critical feature information in fingerprints, directional field, and minutiae, as supervision information. FingerRT inherits the denoising ability of the fingerprint enhancement networks and the powerful generative ability of the Vision Transformer architecture, enabling high-quality and robust fingerprint information recovery. By imposing constraints at multiple levels, including fingerprint features, fingerprint images, and multi-stage generation, FingerRT can complete fingerprint information accurately and effectively. Experiments demonstrate that FingerRT significantly enhances fingerprint recognition accuracy after recovery across various fingerprint datasets, including rolled, snapped, and latent fingerprints.

The security of Automated Fingerprint Identification Systems (AFIS) heavily relies on the performance of the Fingerprint Presentation Attack Detection (FPAD) methods. However, the difficulty of FPAD lies in how to have strong robustness and generalization to unseen spoof fingerprints. To address this issue, we propose a novel FPAD framework with tailored Supervised Contrastive Learning (SupCon) and KNN-based OOD detection (KNN-OOD) method. We tailor the SupCon to better constrain the distribution of learned features by incorporating dynamic feature and label queues into SupCon and actively mining positive samples from the queues. In FPAD, we consider fingerprints with the same PAD label as intra-class, while those with different labels as inter-class. The tailored SupCon makes intra-class features more compact and inter-class features more dispersed. Utilizing the compact live fingerprint feature distribution, during the testing phase, we employ KNNOOD as an alternative to commonly used classification approaches. Since this approach does not rely on the distribution of trainset spoof fingerprints, it consistently achieves outstanding results even for unseen spoof fingerprints. Experiment results demonstrate that our proposed FPAD-SupCon framework achieves state-of-the-art performance on LivDet 2019 and LivDet 2021 datasets.

Latent fingerprint segmentation is a crucial process in contemporary biometric systems utilized in criminal investigations and security applications. Accurately segmenting the fingerprint region from the background noise and artifacts, which can be challenging due to the complexity of the surrounding environment, is the primary goal of this process. Although various methodologies, including binarization-based, texture-based, and deep learning-based segmentation approaches have been proposed, they are often limited by environmental noise and a scarcity of annotated data, resulting in a low segmentation accuracy rate. In this paper, we propose FingerSTR (Finger Segmentation Transformer), a fully Transformer-based latent fingerprint segmentation network, and introduce a new teacher-student training methodology to achieve more precise and robust segmentation results without requiring manual annotation. Based on experimental results of latent fingerprint database NIST SD27, FingerSTR surpasses both deep-learning algorithms and handcraft methods, achieving state-of-the-art performance in the latent fingerprint segmentation task.

Latent fingerprints play a vital role in forensic investigations. However, accurately estimating their orientation field can be challenging due to complex noise or overlapping fingerprint regions. In this paper, we propose a method to identify and correct these regions in the orientation field estimation. Specifically, our method comprises two networks: the first is an orientation field estimation network that outputs the initial orientation field, segment, and quality map, which determines the low-quality regions, including overlapping fingerprints and unclear ridge areas. The second network refills the orientation field in low-quality regions using inpainting techniques. This effectively handles unclear ridges and overlapping fingerprints, which can disrupt orientation field estimation. We assess our method using the NIST SD 27 dataset and demonstrate superior performance compared to existing state-of-the-art latent orientation field estimation methods, achieving the average root mean square deviation of 11.20.

With compact and discriminative fingerprint representation, fingerprint indexing can effectively reduce the search space and improve the efficiency in large-scale fingerprint identification. Previous fixed-length fingerprint representations do not combine the global and minutiae local information well, leading to unsatisfactory results. In this paper, we utilize an end-to-end network to extract Minutiae-aware fingerprint RepresentationS (MaRs) that consider both the global and minutiae local information. The proposed fingerprint representation is weighted aggregated by the output feature map of the network. We hope that not only the global pattern but also the matching minutia-centered regions are similar for the paired fingerprints. We impose constraints among proposed fingerprint representations and constraints among minutia local representations aggregated from each minutia-centered region. The latter constraint strengthens the similarity of matched minutiae local regions. Experimental results show that our minutiaeaware global representation outperforms previous methods in fingerprint indexing on two benchmarks and exhibits strong indexing robustness with a 100k expanded database.

In mobile devices, the limited area of fingerprint sensors brings demand of partial fingerprint matching. Existing fingerprint authentication algorithms are mainly based on minutiae matching. However, their accuracy degrades significantly for partial-to-partial matching due to the lack of minutiae. Optical fingerprint sensor can capture very high-resolution fingerprints (2000dpi) with rich details as pores, scars, etc. These details can cover the shortage of minutiae insufficiency. In this paper, we propose a novel matching algorithm for such fingerprints, namely Deep Joint KNN-Triplet Embedding, by making good use of these subtle features. Our model employs a deep convolutional neural network (CNN) with a well-designed joint loss to project raw fingerprint images into an Euclidean space. Then we can use L2-distance to measure the similarity of two fingerprints. Experiments indicate that our model outperforms several state-of-the-art approaches.

The ridges of fingerprint contain enormous discriminative information for fingerprint indexing, however it is hard to depict the structure of ridges for rule-based methods because of nonlinear distortion. This paper investigates to represent the structure of ridges by Deep Convolutional Neural Network (DCNN). The indexing approach partitions the fingerprint image into increasing fine sub-region and extracts feature from each sub-region by DCNN, forming pyramid deep convolutional feature, to represent the global patterns and local details (especially minutiae). Extensive experimental results show that the proposed method achieves better performance on accuracy and efficiency than other prominent indexing approaches. Finally, occlusion sensitivity, visualization and fingerprint reconstruction techniques are employed to explore which attributes of ridges are described in deep convolutional feature.

Minutiae play a major role in fingerprint identification. Extracting reliable minutiae is difficult for latent fingerprints which are usually of poor quality. As the limitation of traditional handcrafted features, a fully convolutional network (FCN) is utilized to learn features directly from data to overcome complex background noises. Raw fingerprints are mapped to a correspondingly-sized minutia-score map with a fixed stride. And thus a large number of minutiae will be extracted through a given threshold. Then small regions centering at these minutia points are entered into a convolutional neural network (CNN) to reclassify these minutiae and calculate their orientations. The CNN shares convolutional layers with the fully convolutional network to speed up. 0.45 second is used on average to detect one fingerprint on a GPU. On the NIST SD27 database, we achieve 53% recall rate and 53% precise rate that outperform many other algorithms. Our trained model is also visualized to show that we have successfully extracted features preserving ridge information of a latent fingerprint.

Minutiae extraction is of critical importance in automated fingerprint recognition. Previous works on rolled/slap fingerprints failed on latent fingerprints due to noisy ridge patterns and complex background noises. In this paper, we propose a new way to design deep convolutional network combining domain knowledge and the representation ability of deep learning. In terms of orientation estimation, segmentation, enhancement and minutiae extraction, several typical traditional methods performed well on rolled/slap fingerprints are transformed into convolutional manners and integrated as an unified plain network. We demonstrate that this pipeline is equivalent to a shallow network with fixed weights. The network is then expanded to enhance its representation ability and the weights are released to learn complex background variance from data, while preserving end-to-end differentiability. Experimental results on NIST SD27 latent database and FVC 2004 slap database demonstrate that the proposed algorithm outperforms the state-of-the-art minutiae extraction algorithms. Code is made publicly available at: https://github.com/felixTY/FingerNet.

Fingerprint sensors on mobile devices commonly have limited area, which results in partial fingerprints. Optical sensor can capture fingerprints at very high resolution (2000ppi) with abundant details like pores, incipients, etc. It is quite crucial to develop effective partial-to-partial high-resolution fingerprint matching algorithms. Existing fingerprint matching methods are mainly minutiae-based, with fusion of different levels of features. Their accuracy degrades significantly in our application due to minutiae insufficiency and detection error. In this paper, we propose a novel representation for partial high-resolution fingerprint, named Deep Dense Multi-level feature (DDM). We train a deep convolutional neural network that can extract discriminative features inside any local fingerprint block with certain size. We find that not only minutiae but most local blocks contain sufficient features. Moreover, we analyze DDM and find that it contains multi-level information. When utilizing DDM for partial-to-partial matching, we first extract features block by block through a fully convolutional network, next match the two sets of features pairwise exhaustively, and then select the bi-directional best matches to compute matching score. Experiments indicate that our method outperforms several state-of-the-art approaches.

Aimed at the problem that linear discriminative analysis algorithms for face recognition usually miss discriminative information when reducing dimensions, and the problem that it is difficult to make full use of discriminative information both in rank space and null space at the same time, this paper proposes a novel method to construct a new subspace called complete discriminative subspace and its discriminant matrix without losing any discriminative information contained in original space. The dimension of the new subspace is much lower and the constructing procedure is simple and costless. The experimental results demonstrate that this method has better performance and efficiency than standard discriminative analysis algorithms for face recognition.

We propose a low-rank subspace recovery and image denoising method for face recognition. Traditional subspace methods commonly assume that face images from a single class lie on a low-rank subspace. However, due to shadows, specularities, occlusion and corruption, real face images seldom reveal such low-rank structure. To address this problem, we cast the problem of recovering face subspace from noisy images as a problem of recovering a low-rank matrix with sparse error of arbitrary large magnitude. By using the recent breakthroughs in convex optimization, we can exactly recover the subspaces from corrupted facial data. We apply this method to two well-known subspace methods: nearest subspace and sparse representation face recognition. The results show that our method is efficient in recovering the low-rank face subspaces by removing the noise in the training images, thus significantly improve the robustness of these methods in the presence of occlusion and corruption in both train and test images.

Pose variance is one of the most challenging problem to 2D face recognition. In this paper, a novel frontal view face synthesizing strategy is introduced to improve the performance of traditional face recognition methods on non-frontal view input images. Given several non-frontal input faces, our minimum bending synthesizing strategy automatically picks up and merges information, to realize most natural frontal view face synthesizing. It is shown by experiments that our strategy could effectively reduce the influence of pose variance to face recognition, and rather than traditional landmark based approaches, our strategy does not require perfect landmark locating results.

In this paper, a new non-linear distortion robust fingerprint feature named Minutiae Phase Difference Feature (MPDF) is proposed, and used to measure the similarity of fingerprint local structures. Then a novel minutiae expanding method based on Moving Least Squares (MLS) transformation is introduced to regain the lost matched minutiae pairs after obtaining a few valid pairs. Finally a score evaluation strategy based on convex hull of transformed minutiae is adopted, which is more reasonable for partial overlapping fingerprint. Experiments of proposed algorithm conducted on FVC2004 database, comparing with other matching methods, show that the MPDF feature and proposed algorithm have promising performances.

In this paper, a novel method of fingerprint minutiae extraction on grey-scale images is proposed based on the Gabor phase field. The novelty of our approach is that the proposed algorithm is performed on the transform domain, i.e. the Gabor phase field of the fingerprint image. This is different from most existing minutiae extraction methods, in which the minutiae are usually extracted from the binarized and thinned fingerprint image. Experimental results on benchmark data sets demonstrate that the proposed algorithm has promising performances.

We present a novel online algorithm called online AdaBoost ECOC (error-correcting output codes) for image classification problems. In recent years, AdaBoost is very successful in many domains such as object detection in images and videos. It is a representative large margin classifier for binary classification problems and is efficient for on-line learning. However, image classification is a typical multi-class problem. It is difficult to use AdaBoost here, especially in an online version of image classification problem. In this paper, we combine online AdaBoost and ECOC algorithm to solve online multi-class image classification problems. We perform online AdaBoost ECOC on MNIST handwritten digit, ORL face and UCI image database. The results show our algorithm's accuracy and robustness.

This paper introduces a novel method to automatically synthesizing frontal view face images from more than one ill-posed face images. Rather than relying on a uniform warping function to the whole image, our deformation strategy is specific to each pixel. And rather than relying on inaccurate pose estimation, the contribution of each input image is determined automatically to minimize the non-rigid deformation contained in frontal view synthesizing. It can be learned intuitively from our experimental results that this synthesizing strategy is effective, and under the FERET evaluation methodology, it is proved that this strategy is helpful to improve the performance of face recognition algorithms.

This paper introduces a novel shape model, Sparse Representation Shape Model (SRSM). Rather than for modeling specific deformable shapes, this model is specially designed for shape segmentation and matching. This model is utilized under the framework of Active Shape Models (ASM). Unlike the Linear Point Distribution Model utilized by original ASM, which relies on obscure statistical boundary to do shape regularization, SRSM distinctly distinguishes valid shape information and errors contained in input candidate shape from structure and by making use of sparse representation, SRSM could acquire the maximum valid shape information, and hence could achieve optimal shape regularization. Further-more, through exploiting the reliability information of each landmark, SRSM can be improved further to form Weighted SRSM, which is much more evident and accurate.

In this paper, we present a novel online face recognition approach for video stream called online boosting OC (output code). Recently, boosting was successfully used in many study fields such as object detection and tracking. It is one kind of large margin classifiers for binary classification problems and also efficient for on-line learning. However, face recognition is a typical multi-class problem. Hence, it is difficult to use boosting in face recognition, especially in an online version. In our work, we combine online boosting and OC algorithm to solve real-time online multi-class classification problems. We perform online boosting OC on real-world experiments: face recognition in continuous video stream, and the results show that our algorithm is accurate and robust.

The IMage Euclidean Distance (IMED) is a class of image metrics, in which the spatial relationship between pixels is taken into consideration. It was shown that calculating the IMED of two images is equivalent to performing a linear transformation called Standardizing Transform (ST) and then followed by the traditional Euclidean distance. However, while the IMED is invariant to image shift, the ST is not a Shift-Invariant (SI) filter. This left as an open problem whether IMED is equivalent to SI transformation plus traditional Euclidean distance. In this paper, we give a positive answer to this open problem. Specifically, for a wider class of metrics, including IMED, we construct closed-form SI transforms. Based on the SI metric-transform connection, we next develop an image metric learning algorithm by learning a metric filter in the transform domain. This is different from all previous metric approaches. Experimental results on benchmark datasets demonstrate that the learned image metric has promising performances.

We study the problem of classification when only a dissimilarity function between objects is accessible. That is, data samples are represented not by feature vectors but in terms of their pairwise dissimilarities. We establish sufficient conditions for dissimilarity functions to allow building accurate classifiers. The theory immediately suggests a learning paradigm: construct an ensemble of simple classifiers, each depending on a pair of examples; then find a convex combination of them to achieve a large margin. We next develop a practical algorithm referred to as dissimilarity-based boosting (DBoost) for learning with dissimilarity functions under theoretical guidance. Experiments on a variety of databases demonstrate that the DBoost algorithm is promising for several dissimilarity measures widely used in practice.

Using the kernel trick idea and the kernels-as-features idea, we can construct two kinds of nonlinear feature spaces, where linear feature extraction algorithms can be employed to extract nonlinear features. In this correspondence, we study the relationship between the two kernel ideas applied to certain feature extraction algorithms such as linear discriminant analysis, principal component analysis, and canonical correlation analysis. We provide a rigorous theoretical analysis and show that they are equivalent up to different scalings on each feature. These results provide a better understanding of the kernel method.

We study the problem of learning a classification task in which only a dissimilarity function of the objects is accessible. That is, data are not represented by feature vectors but in terms of their pairwise dissimilarities. We investigate the sufficient conditions for dissimilarity functions to allow building accurate classifiers. Our results have the advantages that they apply to unbounded dissimilarities and are invariant to order-preserving transformations. The theory immediately suggests a learning paradigm: construct an ensemble of decision stumps each depends on a pair of examples, then find a convex combination of them to achieve a large margin. We next develop a practical algorithm called Dissimilarity based Boosting (DBoost) for learning with dissimilarity functions under the theoretical guidance. Experimental results demonstrate that DBoost compares favorably with several existing approaches on a variety of databases and under different conditions.

A unified probabilistic framework (UPF) of partitional clustering algorithms is proposed based on Penalized Maximum Likelihood. Besides Gaussian Mixture model methods, many popular clustering methods, such as Fuzzy c-Means Algorithm (FCM), Attribute Means Clustering (AMC), General c-Means Clustering (GCM), and Deterministic Annealing (DA) Clustering can be explained as special cases within UPF. Furthermore, this UPF framework provides a general approach to design comparatively stable and effectively regularized clustering algorithms.

Fundamental limits of reconstruction-based superresolution were proposed in Z. Lin and H. Y. Shum (IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 1, pp. 83-97, Jan. 2004). In this note, we point out that their proof is incomplete.

Principal component analysis (PCA) and linear discriminant analysis (LDA) are two important feature extraction methods and have been widely applied in a variety of areas. A limitation of PCA and LDA is that when dealing with image data, the image matrices must be first transformed into vectors, which are usually of very high dimensionality. This causes expensive computational cost and sometimes the singularity problem. Recently two methods called two-dimensional PCA (2DPCA) and two-dimensional LDA (2DLDA) were proposed to overcome this disadvantage by working directly on 2-D image matrices without a vectorization procedure. The 2DPCA and 2DLDA significantly reduce the computational effort and the possibility of singularity in feature extraction. In this paper, we show that these matrices based 2-D algorithms are equivalent to special cases of image block based feature extraction, i.e., partition each image into several blocks and perform standard PCA or LDA on the aggregate of all image blocks. These results thus provide a better understanding of the 2-D feature extraction approaches.

We present a new Euclidean distance for images, which we call IMage Euclidean Distance (IMED). Unlike the traditional Euclidean distance, IMED takes into account the spatial relationships of pixels. Therefore, it is robust to small perturbation of images. We argue that IMED is the only intuitively reasonable Euclidean distance for images. IMED is then applied to image recognition. The key advantage of this distance measure is that it can be embedded in most image classification techniques such as SVM, LDA, and PCA. The embedding is rather efficient by involving a transformation referred to as Standardizing Transform (ST). We show that ST is a transform domain smoothing. Using the Face Recognition Technology (FERET) database and two state-of-the-art face identification algorithms, we demonstrate a consistent performance improvement of the algorithms embedded with the new metric over their original versions.

In this article, a novel thinning algorithm, based on a time-reversed heat conduction model, is proposed. The image is viewed as a thermal conductor and the thinning task is then considered as an inverse process of heat conduction. Given an image, the direction map of heat conduction is first computed, and then time-reversed heat conduction is simulated. The result turns out to be a thinned pattern. The algorithm can be applied to gray-scale or binary images.

We adopt a local Fourier transform as a texture representation scheme and derive eight characteristic maps for describing different aspects of cooccurrence relations of image pixels in each channel of the (SVcosH, SVsinH, V) color space. Then we calculate the first and second moments of these maps as a representation of the natural color image pixel distribution, resulting in a 48-dimensional feature vector. The novel low-level feature is named color texture moments (CTM), which can also be regarded as a certain extension to color moments in eight aspects through eight orthogonal templates. Experiments show that this new feature can achieve good retrieval performance for CBIR.

In this paper, we develop an improved algorithm to recover and correct polyhedron from a given line drawing. It can immediately recover the polyhedron even if image data are incorrect due to vertex-position errors. Many experiments have been done to demonstrate the algorithm to be effective and efficient and show that the speed of recovery and correction is greatly improved and one of experiments is given.