Recent advances in speech-driven facial animation have attracted significant interest across computer graphics, human–computer interaction systems, and immersive virtual reality applications. However, existing methods remain constrained by dependencies on specific reference videos or proprietary face mesh structures, limiting their applicability across diverse production pipelines and reducing compatibility with industry-standard animation workflows. To overcome these fundamental limitations in generalization and deployment flexibility, we propose Speech2Blend—an end-to-end hybrid convolutional-recurrent network that directly learns nonlinear speech-to-blendshape parameter mappings. This novel approach enables markerless speech-driven facial animation generation without restrictive inputs like video references or specialized facial rigs. Trained on the largest available digital human dataset (BEAT) and rigorously evaluated using three benchmark datasets with photorealistic visualization tools, Speech2Blend achieves state-of-the-art performance. It delivers superior audio-visual synchronization through learned temporal dynamics and reduces lip vertex error by 30% compared to existing baseline methods. These advances significantly lower production costs for virtual human speech animation while enabling cross-platform compatibility with common game engines and animation software.

Deep learning-based medical ultrasound image segmentation holds significant potential for clinical applications but faces challenges such as speckle noise, irregular shapes, and acoustic artifacts. Current CNN and Transformer-based methods suffer from limited receptive fields and high computational complexity, while emerging Mamba models struggle with multi-scale feature integration. To address these limitations, we propose BCMamba, a novel architecture that integrates CNNs with Mamba through two core components: the TiFusion module for multiscale feature filtering and integration, and the Mixed Weight and Fine Feature Decoder (MWFFD) for resolving semantic discrepancies in spatial features. A dynamic boundary-sensing hybrid loss function further mitigates boundary ambiguity and optimization conflicts. Extensive experiments on four ultrasound datasets (BUSI, STU, DDTI, and SZU-BCH-TUS983) demonstrate our method's superiority, achieving state-of-the-art results.

Adversarial attacks present a critical challenge to deep neural networks' robustness, particularly in transfer scenarios across different model architectures. However, the transferability of adversarial attacks faces a fundamental dilemma between Exploitation (maximizing attack potency) and Exploration (enhancing cross-model generalization). Traditional momentum-based methods over-prioritize Exploitation, i.e., higher loss maxima for attack potency but weakened generalization (narrow loss surface). Conversely, recent methods with inner-iteration sampling over-prioritize Exploration, i.e., flatter loss surfaces for cross-model generalization but weakened attack potency (suboptimal local maxima). To resolve this dilemma, we propose a simple yet effective Gradient-Guided Sampling (GGS), which harmonizes both objectives through guiding sampling along the gradient ascent direction to improve both sampling efficiency and stability. Specifically, based on MI-FGSM, GGS introduces inner-iteration random sampling and guides the sampling direction using the gradient from the previous inner-iteration (the sampling's magnitude is determined by a random distribution). This mechanism encourages adversarial examples to reside in balanced regions with both flatness for cross-model generalization and higher local maxima for strong attack potency. Comprehensive experiments across multiple DNN architectures and multimodal large language models (MLLMs) demonstrate the superiority of our method over state-of-the-art transfer attacks. Code is made available at https://github.com/anuin-cat/GGS.

Internal fingerprint reconstruction is critical for bridging traditional fingerprint recognition with Optical Coherence Tomography (OCT)-based techniques. However, current reconstructed internal fingerprints often suffer from low ridge-valley contrast, noise interference, and ridge adherence issues. Traditional fingerprint enhancement techniques address these challenges but involve reconstructing 3D OCT fingerprints into 2D internal fingerprints, followed by enhancement. This two-step approach leads to module inconsistencies and difficulties in parameter setting during the enhancement process. To overcome these limitations, we for the first time propose a novel method that directly reconstructs binarized internal fingerprints. The proposed method employs an image region regression module that directly treats ridge blocks within B-scan images as regional units for regression, yielding 1D feature vectors representing ridges and valleys. Additionally, leveraging the continuity of information between adjacent B-scan images, a window adjustment function is introduced to refine the regression values, ensuring more stable binarized internal fingerprints. Experiments were conducted on publicly available OCT fingerprint benchmark datasets to compare the minutiae extraction and matching performance. The binarized internal fingerprints obtained by the proposed method achieved the highest mean NFIQ2 score. Based on the NBIS software compared to existing OCT internal fingerprint reconstruction methods, the proposed method achieved the lowest Equal Error Rate (EER) of 0.78%. In addition, compared to traditional fingerprint enhancement methods, the proposed method attained the highest F1-score for minutiae extraction at 72.39%. It also achieved the lowest EER and represented a 37.1% reduction compared to the best existing result.

Vehicle re-identification plays a key role in intelligent transportation systems by supporting critical applications such as traffic flow optimization and public safety. With the introduction of triplet loss, the greatly enhanced vehicle discriminative ability brings more possibilities for the practical applications of vehicle re-identification. However, traditional triplet loss may not be fully suited for vehicle re-identification due to the risk of falling into local minima caused by solely selecting hard triplets. To address this problem, a Knowledge-Driven Multi-Branch Interaction Network (KMINet) is proposed which emphasizes robustness, feature diversity and feature fusion. At first, a knowledge-driven triplet mining module is introduced, which dynamically varies the selection of positive and negative samples. This module enables the network to learn the significance of features from a diverse range of triplets. Then, to tackle the challenge of viewpoint variation, a global and local multi-branch structure is designed to leverage both transformer and convolutional blocks to extract global context features and local detail features. This ensures that our model captures comprehensive vehicle characteristics. Finally, we incorporate an attention transfer module within the multi-branch structure to achieve information interaction and reduce feature redundancy across different branches. Our method outperforms state-of-the-art models on the VeRi-776 dataset, achieving 85.71% mAP and 97.73% CMC1. Moreover, the competitive performance is demonstrated on both VehicleID and VERI-Wild datasets, with mAP scores of 92.50% and 88.56%, respectively.

Optical coherence tomography (OCT), as a non-invasive and high-resolution three-dimensional imaging technology, can capture biological tissue structure information under the skin of fingertips. This structure information facilitates stronger anti-spoofing capability of automatic fingerprint recognition systems (AFRSs), and the reconstructed internal fingerprint images based on the structural information are more robust against poor skin conditions. Various internal fingerprint reconstruction methods have been proposed, but these approaches often ignore the continuity of spatial structure information, have a large number of model parameters and are sensitive to noise. Specific to these problems, this paper proposes a lightweight and noise-robust point detection network (LNPDN) to reconstruct internal fingerprints. At first, by combining the ShuffleNet with the temporal shift module and self-attention, the continuity of spatial information is considered. Meanwhile, the previous refined tissue structural region segmentation task, which is highly affected by noise, is transformed into an easy noise-robust feature point detection mission. Then, these detected points are synthesized into a curve to represent the upper envelope of the viable epidermis by linear interpolation. Finally, internal fingerprint image is reconstructed by averaging those pixel values at a certain depth range below the envelope. The experimental results show the proposed feature point extraction model for the central vertex of ridge blocks reaches the F1-score value of 93.911%, and the average minimum point-segment distance between the proposed curve and the target curve is 1.475. It demonstrates that the proposed model can well extract the central vertex of the ridge blocks and the curve can reflect the location of the viable epidermis. We also compared the recognition capabilities of internal fingerprints extracted from 2138 OCT fingerprint volume data on the public OCT fingerprint benchmark dataset. Our method achieves the lowest equal error rate of 0.167%, with a relative reduction of 60.91% compared with state-of-the-art reconstruction methods.

Biometric systems are vulnerable to presentation attacks (PAs) performed using various PA instruments (PAIs). Even though there are numerous PA detection (PAD) techniques based on both deep learning and hand-crafted features, the generalization of PAD for unknown PAI is still a challenging problem. In this work, we empirically prove that the initialization of the PAD model is a crucial factor for generalization, which is rarely discussed in the community. Based on such observation, we proposed a self-supervised learning-based method, denoted as DF-DM. Specifically, DF-DM is based on a global-local view coupled with de-folding and de-mixing to derive the task-specific representation for PAD. During de-folding, the proposed technique will learn region-specific features to represent samples in a local pattern by explicitly minimizing the generative loss. While de-mixing drives detectors to obtain the instance-specific features with global information for more comprehensive representation by minimizing the interpolation-based consistency. Extensive experimental results show that the proposed method can achieve significant improvements in terms of both face and fingerprint PAD in more complicated and hybrid datasets when compared with the state-of-the-art methods. When training in CASIA-FASD and Idiap Replay-Attack, the proposed method can achieve an 18.60% equal error rate (EER) in OULU-NPU and MSU-MFSD, exceeding the baseline performance by 9.54%. The source code of the proposed technique is available at https://github.com/kongzhecn/dfdm.

Facial expression editing has a wide range of applications, such as emotion detection, human-computer interaction, and social entertainment. However, existing expression editing methods either fail to allow for fine-grained editing, resulting in unnatural and unrealistic facial expressions, or generate artifacts and blurs, leading to poor image quality. In this paper, we propose a novel framework called StyleAU, which is based on StyleGAN and facial action units, to address these problems. Our framework leverages the pre-trained StyleGAN prior knowledge to enable action unit editing of the face in the StyleGAN latent space, allowing precise expression editing. In addition, we use an encoder to extract multi-scale content features to achieve high-fidelity image reconstruction. Our approach qualitatively and quantitatively outperforms competing methods for action unit manipulation and expression editing.

Recent studies have shown the vulnerability of CNNs under perturbation noises, which is partially caused by the reason that the well-trained CNNs are too biased toward the object texture, i.e., they make predictions mainly based on texture cues. To reduce this texture-bias, current studies resort to learning augmented samples with heavily perturbed texture to make networks be more biased toward relatively stable shape cues. However, such methods usually fail to achieve real shape-biased networks due to the insufficient diversity of the shape cues. In this paper, we propose to augment the training dataset by generating semantically meaningful shapes and samples, via a shape deformation-based online augmentation, namely as SDbOA. The samples generated by our SDbOA have two main merits. First, the augmented samples with more diverse shape variations enable networks to learn the shape cues more elaborately, which encourages the network to be shape-biased. Second, semantic-meaningful shape-augmentation samples could be produced by jointly regularizing the generator with object texture and edge-guidance soft constraint, where the edges are represented more robustly with a self information guided map to better against the noises on them. Extensive experiments under various perturbation noises demonstrate the obvious superiority of our shape-bias-motivated model over the state of the arts in terms of robustness performance. Code is available at https://github.com/C0notSilly/-ICCV-23-Edge-Deformation-based-Online-Augmentation.

The vulnerability of automated fingerprint recognition systems (AFRSs) to presentation attacks (PAs) promotes the vigorous development of PA detection (PAD) technology. However, PAD methods have been limited by information loss and poor generalization ability, resulting in new PA materials and fingerprint sensors. This article thus proposes a global-local model-based PAD (RTK-PAD) method to overcome those limitations to some extent. The proposed method consists of three modules, called: 1) the global module; 2) the local module; and 3) the rethinking module. By adopting the cut-out-based global module, a global spoofness score predicted from nonlocal features of the entire fingerprint images can be achieved. While by using the texture in-painting-based local module, a local spoofness score predicted from fingerprint patches is obtained. The two modules are not independent but connected through our proposed rethinking module by localizing two discriminative patches for the local module based on the global spoofness score. Finally, the fusion spoofness score by averaging the global and local spoofness scores is used for PAD. Our experimental results evaluated on LivDet 2017 show that the proposed RTK-PAD can achieve an average classification error (ACE) of 2.28% and a true detection rate (TDR) of 91.19% when the false detection rate (FDR) equals 1.0%, which significantly outperformed the state-of-the-art methods by ~10% in terms of TDR (91.19% versus 80.74%).

Due to the diversity of attack materials, fingerprint recognition systems (AFRSs) are vulnerable to malicious attacks. It is thus important to propose effective fingerprint presentation attack detection (PAD) methods for the safety and reliability of AFRSs. However, current PAD methods often exhibit poor robustness under new attack types settings. This paper thus proposes a novel channel-wise feature denoising fingerprint PAD (CFD-PAD) method by handling the redundant noise information ignored in previous studies. The proposed method learns important features of fingerprint images by weighing the importance of each channel and identifying discriminative channels and “noise” channels. Then, the propagation of “noise” channels is suppressed in the feature map to reduce interference. Specifically, a PA-Adaptation loss is designed to constrain the feature distribution to make the feature distribution of live fingerprints more aggregate and that of spoof fingerprints more disperse. Experimental results evaluated on the LivDet 2017 dataset showed that the proposed CFD-PAD can achieve 2.53% average classification error (ACE) and a 93.83% true detection rate when the false detection rate equals 1.0% (TDR@FDR=1%). Also, the proposed method markedly outperforms the best single-model-based methods in terms of ACE (2.53% vs. 4.56%) and TDR@FDR=1%(93.83% vs. 73.32%), which demonstrates its effectiveness. Although we have achieved a comparable result with the state-of-the-art multiple-model-based methods, there still is an increase in TDR@FDR=1% from 91.19% to 93.83%. In addition, the proposed model is simpler, lighter and more efficient and has achieved a 74.76% reduction in computation time compared with the state-of-the-art multiple-model-based method.The source code is available athttps://github.com/kongzhecn/cfd-pad.

Learning-based classifiers are susceptible to adversarial examples. Existing defence methods are mostly devised on individual classifiers. Recent studies showed that it is viable to increase adversarial robustness by promoting diversity over an ensemble of models. In this paper, we propose adversarial defence by encouraging ensemble diversity on learning high-level feature representations and gradient dispersion in simultaneous training of deep ensemble networks. We perform extensive evaluations under white-box and black-box attacks including transferred examples and adaptive attacks. Our approach achieves a significant gain of up to 52% in adversarial robustness, compared with the baseline and the state-of-the-art method on image benchmarks with complex data scenes. The proposed approach complements the defence paradigm of adversarial training, and can further boost the performance. The source code is available at https://github.com/ALIS-Lab/AAAI2021-PDD.

Though 2D face recognition (FR) has achieved great success due to powerful 2D CNNs and large-scale training data, it is still challenged by extreme poses and illumination conditions. On the other hand, 3D FR has the potential to deal with aforementioned challenges in the 2D domain. However, most of available 3D FR works transform 3D surfaces to 2D maps and utilize 2D CNNs to extract features. The works directly processing point clouds for 3D FR is very limited in literature. To bridge this gap, in this paper, we propose a light-weight framework, named PointFace, to directly process point set data for 3D FR. Inspired by contrastive learning, our PointFace use two weight-shared encoders to directly extract features from a pair of 3D faces. A feature similarity loss is designed to guide the encoders to obtain discriminative face representations. We also present a pair selection strategy to generate positive and negative pairs to boost training. Extensive experiments on Lock3DFace and Bosphorus show that the proposed PointFace outperforms state-of-the-art 2D CNN based methods.

3D face recognition (FR) is a popular topic in computer vision, since 3D face data is invariant to pose and illumination condition changes which easily affect the performance of 2D FR. Though many 3D solutions have achieved impressive performances on public high-quality 3D face databases, few works concentrate on low-quality 3D FR. As the quality of 3D face acquired by widely used low-cost RGB-D sensors is really low, more robust methods are required to achieve satisfying performance on these 3D face data. To address this issue, we propose a novel two-stage pipeline to improve the performance of 3D FR. In the first stage, we utilize pix2pix network to restore the quality of low-quality face. In the second stage, we launch a multi-quality fusion network (MQFNet) to fuse the features from different qualities and enhance FR performance. Our proposed network achieves the state-of-the-art performance on the Lock3DFace database. Furthermore, extensive controlled experiments are conducted to demonstrate the effectiveness of each model of our network.

Finger vein has attracted substantial attention due to its good security. However, the variability of the finger vein data will be caused by the illumination, environment temperature, acquisition equipment, and so on, which is a great challenge for finger vein recognition. To address this problem, we propose a novel method to design an endto-end deep Convolutional Neural Network (CNN) for robust finger vein recognition. The approach mainly includes an Intrinsic Feature Learning (IFL) module using an auto-encoder network and an Extrinsic Feature Learning (EFL) module based on a Siamese network. The IFL module is designed to estimate the expectation of intra-class finger vein images with various offsets and rotation, while the EFL module is constructed to learn the inter-class feature representation. Then, robust verification is finally achieved by considering the distances of both intrinsic and extrinsic features. We conduct experiments on two public datasets (i.e. SDUMLA-HMT and MMCBNU_6000) and an in-house dataset (MultiView-FV) with more deformation finger vein images, and the equal error rate (EER) is 0.47%, 0.1%, and 1.69% respectively. The comparison against baseline and existing algorithms shows the effectiveness of our proposed method.

The convolutional neural network (CNN) is vulnerable to degraded images with even very small variations (e.g. corrupted and adversarial samples). One of the possible reasons is that CNN pays more attention to the most discriminative regions, but ignores the auxiliary features when learning, leading to the lack of feature diversity for final judgment. In our method, we propose to dynamically suppress significant activation values of CNN by group-wise inhibition, but not fixedly or randomly handle them when training. The feature maps with different activation distribution are then processed separately to take the feature independence into account. CNN is finally guided to learn richer discriminative features hierarchically for robust classification according to the proposed regularization. Our method is comprehensively evaluated under multiple settings, including classification against corruptions, adversarial attacks and low data regime. Extensive experimental results show that the proposed method can achieve significant improvements in terms of both robustness and generalization performances, when compared with the state-of-the-art methods. Code is available at https://github.com/LinusWu/TENET_Training.

Fingerprint recognition has been used for person identification for centuries, and fingerprint features are divided into three levels. The level 3 feature is the fingerprint pore, which can be used to improve the performance of the automatic fingerprint recognition performance and to prevent spoofing in high-resolution fingerprints. Therefore, the accurate extraction of fingerprint pores is quite important. With the development of convolutional neural networks (CNNs), researchers have made great progress in fingerprint feature extraction. However, these supervised-based methods require manually labelled pores to train the network, and labelling pores is very tedious and time consuming because there are hundreds of pores in one fingerprint. In this paper, we design a weakly supervised pore extraction method that avoids manual label processing and trains the network with a noisy label. This method can achieve results comparable with a supervised CNN-based method.

Automated Fingerprint Recognition Systems (AFRSs) have been threatened by Presentation Attack (PA) since its existence. It is thus desirable to develop effective presentation attack detection (PAD) methods. However, the unpredictable PAs make PAD be a challenging problem. This paper proposes a novel One-Class PAD (OCPAD) method for Optical Coherence Technology (OCT) images based fingerprint PA detection. The proposed OCPAD model is learned from a training set only consists of Bonafides (i.e. real fingerprints). The reconstruction error and latent code obtained from the trained auto-encoder network in the proposed model is taken as the basis for the following spoofness score calculation. To get more accurate reconstruction error, we propose an activation map based weighting model to further refine the accuracy of reconstruction error. We test different statistics and distance measures and finally use a decision level fusion to make the final prediction. Our experiments are performed using a dataset with 93200 bonafide scans and 48400 PA scans. The results show that the proposed OCPAD can achieve a True Positive Rate (TPR) of 99.43% when the False Positive Rate (FPR) equals to 10% and a TPR of 96.59% when FPR=5%, which significantly outperformed a feature based approach and a supervised learning based model requiring PAs for training.

As 2D and 3D data present different views of the same face, the features extracted from them can be both complementary and redundant. In this paper, we present a novel and efficient orthogonalization-guided feature fusion network, namely OGF$^2$Net, to fuse the features extracted from 2D and 3D faces for facial expression recognition. While 2D texture maps are fed into a 2D feature extraction pipeline (FE2DNet), the attribute maps generated from 3D data are concatenated as input of the 3D feature extraction pipeline (FE3DNet). The two networks are separately trained at the first stage and frozen in the second stage for late feature fusion, which can well address the unavailability of a large number of 3D+2D face pairs. To reduce the redundancies among features extracted from 2D and 3D streams, we design an orthogonal loss-guided feature fusion network to orthogonalize the features before fusing them. Experimental results show that the proposed method significantly outperforms the state-of-the-art algorithms on both the BU-3DFE and Bosphorus databases. While accuracies as high as 89.05% (P1 protocol) and 89.07% (P2 protocol) are achieved on the BU-3DFE database, an accuracy of 89.28% is achieved on the Bosphorus database. The complexity analysis also suggests that our approach achieves a higher processing speed while simultaneously requiring lower memory costs.

A local feature tensor similarity based deep learning approach is proposed in this paper for 3D face recognition. Once a set of salient points on the 3D mesh are detected, three scale and rotation invariant features are extracted to represent local surface around each salient point. The local features of all the salient points are concatenated to produce a 3rdorder feature tensor to represent a 3D face. Similarity of two 3D faces can thus be measured by a similarity tensor calculated using the two feature tensors. To address the unavailability of large 3D face samples, a feature tensor based data augmentation approach is proposed to augment the number of feature tensors. Experimental results show that the ResNet model trained using the augmented feature tensors achieves the best performance among state of the art competitors, i.e. 99.71% and 96.2% accuracy are achieved for Bosphorus and BU3DFE database, respectively.

This paper proposes an effective pore representation descriptor based on Convolutional Neural Networks (CNNs). We make full use of the diversity and large quantities of sweat pores in fingerprints to learn a deep feature, denoted as DeepPoreID. The DeepPoreID is then used to describe the local feature for each pore and finally integrated into the classical direct pore matching method. Experiments carried on the challenge public high-resolution fingerprint database with small image size of 320 × 240 shows the effectiveness of the proposed DeepPoreID. The results also have shown that the proposed method outperforms other existing state-of-the-art methods in the aspect of recognition accuracy. About ~35% rise in accuracy can be obtained when compared with the best result achieved by existing methods.

High-resolution fingerprint recognition has been a hot topic for many years. Compared with a traditional fingerprint image, a high-resolution fingerprint image can provide more features, such as pores and ridge contours. Introducing these features into fingerprint comparison and recognition can improve the recognition accuracy and reduce the risk of identification errors. This paper proposes a novel method for comparing pores on high-resolution fingerprint images. The method can be divided into two steps. In the first step, fingerprints are aligned using the pixel-category-distance-based data-driven descending algorithm. Traditionally, fingerprints are aligned based on feature points, such as minutiae and singular points. Such alignment methods are not suitable when dealing with partial fingerprints because small overlapping areas often do not contain enough features to guarantee a correct alignment. In this research, the ridges and valleys on fingerprints are used in combination with the orientation field for alignment. The proposed algorithm performs well when aligning both partial and full fingerprints. The common areas between the two images can be estimated based on the alignment result. In the second step, pores lying in the common areas are selected for comparison. To improve the comparison accuracy, pores are compared using local features and spatial relations. A graph comparison algorithm is designed in this step. The experimental results show that the proposed method is more accurate than other state-of-the-art pore comparison algorithms.

This paper proposes an encoder-decoder network to disentangle shape features during 3D face reconstruction from single 2D images, such that the tasks of reconstructing accurate 3D face shapes and learning discriminative shape features for face recognition can be accomplished simultaneously. Unlike existing 3D face reconstruction methods, our proposed method directly regresses dense 3D face shapes from single 2D images, and tackles identity and residual (i.e., non-identity) components in 3D face shapes explicitly and separately based on a composite 3D face shape model with latent representations. We devise a training process for the proposed network with a joint loss measuring both face identification error and 3D face shape reconstruction error. To construct training data we develop a method for fitting 3D morphable model (3DMM) to multiple 2D images of a subject. Comprehensive experiments have been done on MICC, BU3DFE, LFW and YTF databases. The results show that our method expands the capacity of 3DMM for capturing discriminative shape features and facial detail, and thus outperforms existing methods both in 3D face reconstruction accuracy and in face recognition accuracy.

This letter proposes an ensemble neural network (Ensem-NN) for skeleton-based action recognition. The Ensem-NN is introduced based on the idea of ensemble learning, “two heads are better than one.” According to the property of skeleton sequences, we design one-dimensional convolution neural network with residual structure asBase-Net. From entirety to local, from focus to motion, we designed four different subnets based on theBase-Netto extract diverse features. The first subnet is aTwo-stream Entirety Net, which performs on the entirety skeleton and explores both temporal and spatial features. The second is aBody-part Net, which can extract fine-grained spatial and temporal features. The third is anAttention Net, in which a channel-wised attention mechanism can learn important frames and feature channels.Frame-difference Net, as the fourth subnet, aims at exploring motion features. Finally, the four subnets are fused as one ensemble network. Experimental results show that the proposed Ensem-NN performs better than state-of-the-art methods on three widely used datasets.

Three-dimensional (3D) faces are increasingly utilized in many face-related tasks. Despite the promising improvement achieved by 3D face technology, it is still hard to thoroughly evaluate the performance and effect of 3D face technology in real-world applications where variations frequently occur in pose, illumination, expression and many other factors. This is due to the lack of benchmark databases that contain both high precision full-view 3D faces and their 2D face images/videos under different conditions. In this paper, we present such a multi-dimensional face database (namely Multi-Dim) of high precision 3D face scans, high definition photos, 2D still face images with varying pose and expression, low quality 2D surveillance video clips, along with ground truth annotations for them. Based on this Multi-Dim face database, extensive evaluation experiments have been done with state-of-the-art baseline methods for constructing 3D morphable model, reconstructing 3D faces from single images, 3D-assisted pose normalization for face verification, and 3D-rendered multiview gallery for face identification. Our results show that 3D face technology does help in improving unconstrained 2D face recognition when the probe 2D face images are of reasonable quality, whereas it deteriorates rather than improves the face recognition accuracy when the probe 2D face images are of poor quality. We will make Multi-Dim freely available to the community for the purpose of advancing the 3D-based unconstrained 2D face recognition and related techniques towards real-world applications.

Cascaded regression has been recently applied to reconstruct 3D faces from single 2D images directly in shape space, and has achieved state-of-the-art performance. We investigate thoroughly such cascaded regression based 3D face reconstruction approaches from four perspectives that are not well been studied: (1) the impact of the number of 2D landmarks; (2) the impact of the number of 3D vertices; (3) the way of using standalone automated landmark detection methods; (4) the convergence property. To answer these questions, a simplified cascaded regression based 3D face reconstruction method is devised. This can be integrated with standalone automated landmark detection methods and reconstruct 3D face shapes that have the same pose and expression as the input face images, rather than normalized pose and expression. An effective training method is also proposed by disturbing the automatically detected landmarks. Comprehensive evaluation experiments have been carried out to compare to other 3D face reconstruction methods. The results not only deepen the understanding of cascaded regression based 3D face reconstruction approaches, but also prove the effectiveness of the proposed method.

A class-specific random forest (RF) model with cross-correlation constraints is developed for the spectral-spatial hyperspectral image (HSI) classification. The novelties of this letter are as follows: 1) normalization of the spectral feature vector by using cross correlation in the stochastic process and proposal of a spectral-spatial hybrid feature extraction based on the cross-correlation analysis; 2) establishment of an RF classifier model by using class-specific trees (CSTs); and 3) exploration of the performance of the proposed method by comparing with its several traditional classification methods on two real HSI data sets. Further research on the effects of parameter setup, such as the number of CSTs, spectral constraint scale, and size of the spatial neighbor, is discussed in terms of classification accuracy. Experimental results show that the performance of the proposed method is better than that of the traditional methods.

Visual-based traffic sign recognition (TSR) requires first detecting and then classifying signs from captured images. In such a cascade system, classification accuracy is often affected by the detection results. This paper proposes a method for extracting a salient region of traffic sign within a detection window for more accurate sign representation and feature extraction, hence enhancing the performance of classification. In the proposed method, a superpixel-based distance map is firstly generated by applying a signed geodesic distance transform from a set of selected foreground and background seeds. An effective method for obtaining a final segmentation from the distance map is then proposed by incorporating the shape constraints of signs. Using these two steps, our method is able to automatically extract salient sign regions of different shapes. The proposed method is tested and validated in a complete TSR system. Test results show that the proposed method has led to a high classification accuracy (97.11%) on a large dataset containing street images. Comparing to the same TSR system without using saliency-segmented regions, the proposed method has yielded a marked performance improvement (about 12.84%). Future work will be on extending to more traffic sign categories and comparing with other benchmark methods.

Touchless-based fingerprint recognition technology is thought to be an alternative to touch-based systems to solve problems of hygienic, latent fingerprints, and maintenance. However, there are few studies about touchless fingerprint recognition systems due to the lack of a large database and the intrinsic drawback of low ridge-valley contrast of touchless fingerprint images. This paper proposes an end-to-end solution for user authentication systems based on touchless fingerprint images in which a multiview strategy is adopted to collect images and the robust fingerprint feature of touchless image is extracted for matching with high recognition accuracy. More specifically, a touchless multiview fingerprint capture device is designed to generate three views of raw images followed by preprocessing steps including region of interest (ROI) extraction and image correction. The distal interphalangeal crease (DIP)-based feature is then extracted and matched to recognize the human's identity in which part selection is introduced to improve matching efficiency. Experiments are conducted on two sessions of touchless multiview fingerprint image database with 541 fingers acquired about two weeks apart. An EER of ~ 1.7% can be achieved by using the proposed DIP-based feature, which is much better than touchless fingerprint recognition by using scale invariant feature transformation (SIFT) and minutiae features. The given fusion results show that it is effective to combine the DIP-based feature, minutiae, and SIFT feature for touchless fingerprint recognition systems. The EER is as low as ~ 0.5%.

High resolution fingerprint images have been increasingly used in fingerprint recognition. They can provide more fine features (e.g. pores) than standard fingerprint images to improve the recognition accuracy. It is however still an open issue whether or not existing quality assessment methods are suitable for high resolution fingerprint images. This paper compares some typical quality indexes by analyzing the correlation between them and their prediction ability on minutia-based and pore-based high resolution fingerprint recognition accuracy. Experimental results show that the indexes based on ridge orientation are more effective for high resolution fingerprint recognition systems.

This paper proposes an improved direct fingerprint pore matching method. It measures the differences between pores by using the sparse representation technique. The coarse pore correspondences are then established and weighted based on the obtained differences. The false correspondences among them are finally removed by using the weighted RANSAC algorithm. Experimental results have shown that the proposed method can greatly improve the accuracy of existing methods.

Extended fingerprint features such as pores, dots and incipient ridges have been increasingly attracting attention from researchers and engineers working on automatic fingerprint recognition systems. A variety of methods have been proposed to combine these features with the traditional minutiae features. This paper comparatively analyses the parallel and hierarchical fusion approaches on a high resolution fingerprint image dataset. Based on the results, a novel and more effective hierarchical approach is presented for combining minutiae, pores, dots and incipient ridges.