Deep learning has been shown repeatedly to be a successful method of obtaining accurate classifiers. This also applies to orchid identification from digital photographs. However, deep neural networks possess the major weakness of lack of explainability, missing the ability to explain the reasons behind a decision. Nevertheless, most current research regarding automated orchid identification applies this blackbox approach. By contrast, in this paper we propose a new method for trustworthy automated orchid identification combining two complementary methods: deep neural networks and feature-based Bayesian networks, where the Bayesian network is also utilized for providing an explanation of the generated solutions. We use other deep neural networks to extract flower characteristics, the features, from the images which are subsequently fed into the Bayesian network as uncertain evidence. When combining the deep neural network and the Bayesian network as an ensemble classifier, both reaching the same conclusion, an accuracy of 89.4% is achieved, the most trustworthy outcome. With a human-in-the-loop ensemble classifier, validation results are even better, yielding an accuracy of 98.1%. Our approach also exploits the taxonomic knowledge represented in the Bayesian network to provide an explanation of the solutions for every case, reinforcing further trust in the method. The result is an explainable user-in-the-loop ensemble classifier. Providing explainability can help build user trust in a system and may play a major role when it is used as a learning aid for new orchid enthusiasts. Finally, the proposed method may be also of value in many fields other than plant determination.

ABSTRACT We propose a tool that can replace one or more specific facial components in images for face analysis. The tool can replace the texture and shape of facial components such as eyes, nose, and mouth. The source and destination of the components can be real faces or an average face computed from a dataset. A seamless method is applied to smooth the component boundaries after replacement. The tool is developed using the Python language and is available in open source and online, with a web interface. We also provide a desktop version that can manage multiple files or a dataset as input. The tool can, for instance, be used to investigate the contribution of face components to face recognition, face perception analysis, the change of identity, and fun applications. Some illustrative examples are provided.

Abstract Morphing attacks pose a serious threat to automated border control systems by allowing identity documents to be used by multiple individuals, undermining biometric security. To address this, we propose a novel face demorphing framework that leverages the latent space of StyleGAN2. At its core is ReStyle-ID, an encoder network optimized for identity preservation through improved loss functions and targeted training data, enabling accurate and identity-focused inversion. Combined with StyleDemorpher, a face demorphing network trained on a novel DemorphDB dataset with high-quality morph images that simulate realistic and challenging attack scenarios, the framework reconstructs high-resolution demorphed faces and generalizes well to unseen identities and morphing methods. Together, these components overcome key limitations of prior approaches, such as low resolution, poor robustness, and visual artifacts. This work offers a scalable and effective solution for face demorphing and contributes a comprehensive dataset and framework to support future research in biometric security.

Partial face recognition (PFR) is challenging as the appearance of the face changes significantly with occlusion. In particular, these occlusions can be due to any item and may appear in any position that seriously hinders the extraction of discriminative features. Existing methods deal with PFR either by training a deep model with existing face databases containing limited occlusion types or by extracting un-occluded features directly from face regions without occlusions. Limited training data (i.e., occlusion type and diversity) can not cover the real-occlusion situations, and thus training-based methods can not learn occlusion robust discriminative features. The performance of occlusion region-based method is bounded by occlusion detection. Different from limited training data and occlusion region-based methods, we propose to use multi-label attributes for Partial Face Recognition (Attr4PFR). A novel data augmentation is proposed to solve limited training data and generate occlusion attributes. Apart from occlusion attributes, we also include soft biometric attributes and semantic attributes to explore more rich attributes to combat the loss caused by occlusions. To train our Attr4PFR, we propose an implicit attributes loss combined with a softmax loss to enforce Attr4PFR to learn discriminative features. As multi-label attributes are our auxiliary signal in the training phase, we do not need them in the inference. Extensive experiments on public benchmark AR and IJB-C databases show our method is 3% and 2.3% improvement compared to the state-of-the-art.

Abstract Automated image-based plant identification systems are black-boxes, failing to provide an explanation of a classification. Such explanations are seen as being essential by taxonomists and are part of the traditional procedure of plant identification. In this paper, we propose a different method by extracting explicit features from flower images that can be employed to generate explanations. We take the benefit of feature extraction derived from the taxonomic characteristics of plants, with the orchids as an example domain. Feature classifiers were developed using deep neural networks. Two different methods were studied: (1) a separate deep neural network was trained for every individual feature, and (2) a single, multi-label, deep neural network was trained, combining all features. The feature classifiers were tested in predicting 63 orchid species using naive Bayes (NB) and tree-augmented Bayesian networks (TAN). The results show that the accuracy of the feature classifiers is in the range 83-93%. By combining these features using NB and TAN the species can be predicted with an accuracy of 88.9%, which is better than a standard pre-trained deep neural-network architecture, but inferior to a deep learning architecture after fine-tuning of multiple layers. The proposed novel feature extraction method still performs well for identification and is explainable, as opposed to black-box solutions that only aim for the best performance. Graphical abstract

We explore a new method to hide identity information in a facial image from face recognition (FR) systems, while only minimally changing the appearance of the image as perceived by humans. We train a decoder network that reverses the mapping of an FR system and use the dissimilarity score function of this FR system to teach the decoder to return images with as little identity information as possible, while using a visual loss to change the image as little as possible visually. We show that these obfuscation attacks are also successful when the FR system is unknown. We analyse the obfuscated images in latent space and show that our approach as well as an existing method can be easily circumvented by applying the same obfuscation method to the enrolled faces as to the probe images. We suggest an adaptation that can help prevent this circumvention.

Abstract Presentation attack detection (PAD) is used to mitigate the dangers of the weakest link problem in biometric recognition, in which failure modes of one application affect the security of all other applications. Strong PAD methods are therefore a must, and we believe biometric challenge-response protocols (BCRP) form an underestimated part of this ecosystem. In this paper, we conceptualize what BCRPs are, and we propose a descriptive formalism and categorization for working with them. We validate the categorization against existing literature that we classified to be describing BCRPs. Lastly, we discuss how strong BCRPs provide advantages over PAD methods, specifically in the protection of individual applications and the protection of other applications from inadvertent leaks in BCRP applications. We note that research in BCRPs is fragmented, and our intent for the proposed formalism and categorization are to give focus and direction to research efforts into biometric challenge-response protocols.

Abstract In this paper, new insights in the near infrared imaging process used in finger‐vein recognition by developing a physical model are presented. A realistic phantom finger that mimics the living human finger and also includes veins has been developed to validate this model. NIR phantom finger images show that the phantom can emulate the optical properties of a living human finger and can provide ground truth for the locations of the veins. Through physical modeling, it is particularly learned that—besides blood and soft tissue—bone also plays an important role in generating reliable NIR finger‐vein images.

Data and statistics are key to soccer analytics and have important roles in player evaluation and fan engagement. Automatic recognition of soccer events - such as passes and corners - would ease the data gathering process, potentially opening up the market for soccer analytics at non professional clubs. Existing approaches extract events on group level only and rely on television broadcasts or recordings from multiple camera viewpoints. We propose a novel method for the recognition of individual actions and group activities in panoramic videos from a single viewpoint. Three key contributions in the proposed method are (1) player snippets as model input, (2) independent extraction of spatio-temporal features per player, and (3) feature contextuali-sation using zero-padding and feature suppression in graph attention networks. Our method classifies video samples in eight action and eleven activity types, and reaches accuracies above 75% for ten of these classes.

Morphing attacks have posed a severe threat to Face Recognition System (FRS). Despite the number of advancements reported in recent works, we note serious open issues such as independent benchmarking, generalizability challenges and considerations to age, gender, ethnicity that are inadequately addressed. Morphing Attack Detection (MAD) algorithms often are prone to generalization challenges as they are database dependent. The existing databases, mostly of semi-public nature, lack in diversity in terms of ethnicity, various morphing process and post-processing pipelines. Further, they do not reflect a realistic operational scenario for Automated Border Control (ABC) and do not provide a basis to test MAD on unseen data, in order to benchmark the robustness of algorithms. In this work, we present a new sequestered dataset for facilitating the advancements of MAD where the algorithms can be tested on unseen data in an effort to better generalize. The newly constructed dataset consists of facial images from 150 subjects from various ethnicities, age-groups and both genders. In order to challenge the existing MAD algorithms, the morphed images are with careful subject pre-selection created from the contributing images, and further post-processed to remove morphing artifacts. The images are also printed and scanned to remove all digital cues and to simulate a realistic challenge for MAD algorithms. Further, we present a new online evaluation platform to test algorithms on sequestered data. With the platform we can benchmark the morph detection performance and study the generalization ability. This work also presents a detailed analysis on various subsets of sequestered data and outlines open challenges for future directions in MAD research.

Along with the deployment of the Face Recognition Systems (FRS), concerns were raised related to the vulnerability of those systems towards various attacks including morphed attacks. The morphed face attack involves two different face images in order to obtain via a morphing process a resulting attack image, which is sufficiently similar to both contributing data subjects. The obtained morphed image can successfully be verified against both subjects visually (by a human expert) and by a commercial FRS. The face morphing attack poses a severe security risk to the e-passport issuance process and to applications like border control, unless such attacks are detected and mitigated. In this work, we propose a new method to reliably detect a morphed face attack using a newly designed demising framework. To this end, we design and introduce a new deep Multi-scale Context Aggregation Network (MS-CAN) to obtain denoised images, which is subsequently used to determine if an image is morphed or not. Extensive experiments are carried out on three different morphed face image datasets. The Morphing Attack Detection (MAD) performance of the proposed method is also benchmarked against 14 different state-of-the-art techniques using the ISO-IEC 30107-3 evaluation metrics. Based on the obtained quantitative results, the proposed method has indicated the best performance on all three datasets and also on cross-dataset experiments.

The visual world we sense, interpret and interact everyday is a complex composition of interleaved physical entities. Therefore, it is a very challenging task to generate vivid scenes of similar complexity using computers. In this work, we present a scene generation framework based on Generative Adversarial Networks (GANs) to sequentially compose a scene, breaking down the underlying problem into smaller ones. Different than the existing approaches, our framework offers an explicit control over the elements of a scene through separate background and foreground generators. Starting with an initially generated background, foreground objects then populate the scene one-by-one in a sequential manner. Via quantitative and qualitative experiments on a subset of the MS-COCO dataset, we show that our proposed framework produces not only more diverse images but also copes better with affine transformations and occlusion artifacts of foreground objects than its counterparts.

Application‐specific data for certain biometric applications are often not sufficiently available. The authors present a solution for face recognition with limited application‐specific data. Existing methods often use a classifier with convolutional neural networks (CNNs) as feature extractors. The CNNs are trained with massive general (i.e. not application specific) data and the classifier is trained with application‐specific data. Alternatively, the authors propose a combined training strategy to train the classifier on a balanced mixture of general and application‐specific data, such that the recognition performance is maximised. The proposed method largely alleviates the needs for application‐specific data. To prove its effectiveness, they apply the proposed method to low‐resolution face recognition. Specifically, they use the heterogeneous joint Bayesian (HJB) classifier that is capable of comparing features from the same modality but with different characteristics. To further boost performance, the authors augment the training data by pre‐processing it to resemble application‐specific data. They conducted extensive experiments on challenging datasets, namely, SCface and COX. The results show that the proposed method improves the true match rate on SCface at a false match rate of 10% by ∼11% and the true match rate on COX at a false match rate of 1% by ∼12%.

Facial marks have been studied before, either as a complement to face recognition systems or for their suitability as a single biometric modality. In this paper, we use a subset of the FRGCv2 data set (12307 images and 568 subjects) to study the properties of facial marks, their spatial patterns, and classifiers acting upon these patterns. We observe differences between age and ethnic groups in the number of facial marks. Also, facial marks tend to be clustered. We present six forensically relevant aspects with respect to the design and evaluation of classifiers. These aspects help to systematically study factors that influence performance characteristics (discriminating power and calibration loss) of these classifiers. Calibration loss is of particular forensic importance; it essentially measures how well the classifier output can be used as strength of evidence in a court of law. We use various facial mark grids to which the facial mark spatial patterns are assigned. We find that a classifier that utilizes the facial mark grid of a specific subject outperforms all other classifiers. We also observe that the calibration loss of such subject-based classifier indicates that small grid cell sizes should be avoided.

Law enforcement agencies around the world use ridge-based biometrics, especially fingerprints, to fight crime. Fingermarks that are left at a crime scene and identified as potentially having evidential value (EV) in a court of law are recorded for further forensic analysis. Here, we test our evidential value algorithm (EVA) which uses image features trained on forensic expert decisions for 1428 fingermarks to produce an EV score for an image. First, we study the relationship between whether a fingermark is assessed as having EV, either by a human expert or by EVA, and its correct and confident identification by an automatic identification system. In particular, how often does an automatic system achieve identification when the mark is assessed as not having evidential value? We show that when the marks are captured by a mobile phone, correct and confident automatic matching occurs for 257 of the 1428. Of these, 236 were marked as having sufficient EV by experts and 242 by EVA thresholded on equal error rate. Second, we test four relatively challenging ridge-based biometric databases and show that EVA can be successfully applied to give an EV score to all images. Using EV score as an image quality value, we show that in all databases, thresholding on EV improves performance in closed set identification. Our results suggest an EVA application that filters fingermarks meeting a minimum EV score could aid forensic experts at the point of collection, or by flagging difficult latents objectively, or by pre-filtering specimens before submission to an AFIS.

Quality of a pair of facial images is a strong indicator of the uncertainty in decision about identity based on that image pair. In this paper, we describe a Bayesian approach to model the relation between image quality (like pose, illumination, noise, sharpness, etc) and corresponding face recognition performance. Experiment results based on the MultiPIE data set show that our model can accurately aggregate verification samples into groups for which the verification performance varies fairly consistently. Our model does not require similarity scores and can predict face recognition performance using only image quality information. Such a model has many applications. As an illustrative application, we show improved verification performance when the decision threshold automatically adapts according to the quality of facial images.

The increase of the dimensionality of data sets often leads to problems during estimation, which are denoted as the curse of dimensionality. One of the problems of second-order statistics (SOS) estimation in high-dimensional data is that the resulting covariance matrices are not full rank, so their inversion, for example, needed in verification systems based on the likelihood ratio, is an ill-posed problem, known as the singularity problem. A classical solution to this problem is the projection of the data onto a lower dimensional subspace using principle component analysis (PCA) and it is assumed that any further estimation on this dimension-reduced data is free from the effects of the high dimensionality. Using theory on SOS estimation in high-dimensional spaces, we show that the solution with PCA is far from optimal in verification systems if the high dimensionality is the sole source of error. For moderate dimensionality, it is already outperformed by solutions based on euclidean distances and it breaks down completely if the dimensionality becomes very high. We propose a new method, the fixed-point eigenwise correction, which does not have these disadvantages and performs close to optimal.

In this paper we present a new robust approach for 3D face registration to an intrinsic coordinate system of the face. The intrinsic coordinate system is defined by the vertical symmetry plane through the nose, the tip of the nose and the slope of the bridge of the nose. In addition, we propose a 3D face classifier based on the fusion of many dependent region classifiers for overlapping face regions. The region classifiers use PCA-LDA for feature extraction and the likelihood ratio as a matching score. Fusion is realised using straightforward majority voting for the identification scenario. For verification, a voting approach is used as well and the decision is defined by comparing the number of votes to a threshold. Using the proposed registration method combined with a classifier consisting of 60 fused region classifiers we obtain a 99.0% identification rate on the all vs first identification test of the FRGC v2 data. A verification rate of 94.6% at FAR=0.1% was obtained for the all vs all verification test on the FRGC v2 data using fusion of 120 region classifiers. The first is the highest reported performance and the second is in the top-5 of best performing systems on these tests. In addition, our approach is much faster than other methods, taking only 2.5 seconds per image for registration and less than 0.1 ms per comparison. Because we apply feature extraction using PCA and LDA, the resulting template size is also very small: 6 kB for 60 region classifiers.

Verification decisions are often based on second order statistics estimated from a set of samples. Ongoing growth of computational resources allows for considering more and more features, increasing the dimensionality of the samples. If the dimensionality is of the same order as the number of samples used in the estimation or even higher, then the accuracy of the estimate decreases significantly. In particular, the eigenvalues of the covariance matrix are estimated with a bias and the estimate of the eigenvectors differ considerably from the real eigenvectors. We show how a classical approach of verification in high dimensions is severely affected by these problems, and we show how bias correction methods can reduce these problems.

Eigenvalue analysis is an important aspect in many data modeling methods. Unfortunately, the eigenvalues of the sample covariance matrix (sample eigenvalues) are biased estimates of the eigenvalues of the covariance matrix of the data generating process (population eigenvalues). We present a new method based on bootstrapping to reduce the bias in the sample eigenvalues: the eigenvalue estimates are updated in several iterations, where in each iteration synthetic data is generated to determine how to update the population eigenvalue estimates. Comparison of the bootstrap eigenvalue correction with a state of the art correction method by Karoui shows that depending on the type of population eigenvalue distribution, sometimes the Karoui method performs better and sometimes our bootstrap method.

We propose a novel method to correct for arbitrary illumination variation in the face images. The main purpose is to improve recognition results of face images taken under uncontrolled illumination conditions. We correct the illumination variation in the face images using a face shape model, which allows us to estimate the face shape in the face image. Using this face shape, we can reconstruct a face image under frontal illumination. These reconstructed images improve the results in face identification. We experimented both with face images acquired under different controlled illumination conditions in a laboratory and under uncontrolled illumination conditions.

In this paper we investigate the effect of image resolution on the error rates of a face verification system. We do not restrict ourselves to the face recognition algorithm only, but we also consider the face registration. In our face recognition system, the face registration is done by finding landmarks in a face image and subsequent alignment based on these landmarks. To investigate the effect of image resolution we performed experiments where we varied the resolution. We investigate the effect of the resolution on the face recognition part, the registration part and the entire system. This research also confirms that accurate registration is of vital importance to the performance of the face recognition algorithm. The results of our face recognition system are optimal on face images with a resolution of 32 times 32 pixels

Blood pool agents (BPAs) for contrast-enhanced (CE) magnetic-resonance angiography (MRA) allow prolonged imaging times for higher contrast and resolution. Imaging is performed during the steady state when the contrast agent is distributed through the complete vascular system. However, simultaneous venous and arterial enhancement in this steady state hampers interpretation. In order to improve visualization of the arteries and veins from steady-state BPA data, a semiautomated method for artery-vein separation is presented. In this method, the central arterial axis and central venous axis are used as initializations for two surfaces that simultaneously evolve in order to capture the arterial and venous parts of the vasculature using the level-set framework. Since arteries and veins can be in close proximity of each other, leakage from the evolving arterial (venous) surface into the venous (arterial) part of the vasculature is inevitable. In these situations, voxels are labeled arterial or venous based on the arrival time of the respective surface. The evolution is steered by external forces related to feature images derived from the image data and by internal forces related to the geometry of the level sets. In this paper, the robustness and accuracy of three external forces (based on image intensity, image gradient, and vessel-enhancement filtering) and combinations of them are investigated and tested on seven patient datasets. To this end, results with the level-set-based segmentation are compared to the reference-standard manually obtained segmentations. Best results are achieved by applying a combination of intensity- and gradient-based forces and a smoothness constraint based on the curvature of the surface. By applying this combination to the seven datasets, it is shown that, with minimal user interaction, artery-vein separation for improved arterial and venous visualization in BPA CE-MRA can be achieved.

A new method for evaluation of edge detectors, based on the average risk of a decision, is discussed. The average risk is a performance measure well-known in Bayesian decision theory. Since edge detection can be regarded as a compound decision making process, the performance of an edge detector is context dependent. Therefore, the application of average risk to edge detection is non-trivial. The paper describes a method to estimate the probabilities on a number of different types of (context dependent) errors. A weighted sum of these estimated probabilities represents the average risk. The weight coefficients define the cost function. The method is suitable, not only for the comparison of edge operators, but also for the determination of the weaknesses and strengths of a certain edge operator. This is demonstrated with some well-known edge operators.>