Visualization methods have been used to reveal areas of images which influence the decision making of machine learning models, thereby helping to understand and diagnose the learned models and suggest ways to improve their performance. This concept is termed Explainable Artificial Intelligence. In this work, we focus on visualization methods for metric-learning based neural networks. We propose a gradient-weighted extended Excitation Back-Propagation (gweEBP) method that integrates the gradient information during its backpropagation for the accurate investigation of embedding networks. We perform an extensive evaluation of our gweEBP, and seven other visualization methods, on two neural networks, trained for heterogeneous face recognition. The evaluation is performed over two publicly available cross-modality datasets using two evaluation methods termed the “hiding game” and the “inpainting game”. Our experiments showed that the proposed method outperforms the competing methods in both games in most cases. Additionally, our comprehensive study also provides a benchmark for comparing visualization techniques, which may help other researchers develop new techniques and perform comparative studies on them. • We proposed gweEBP for analyzing embedding networks. • Two evaluation protocols were used: the hiding game and the inpainting game. • The inpainting game provided a more reliable metric than the hiding game.

Nondestructive detection methods, based on vibrational spectroscopy, are vitally important in a wide range of applications including industrial chemistry, pharmacy and national defense. Recently, deep learning has been introduced into vibrational spectroscopy showing great potential. Different from images, text, etc. that offer large labeled data sets, vibrational spectroscopic data is very limited, which requires novel concepts beyond transfer and meta learning. To tackle this, we propose a task-enhanced augmentation network (TeaNet). The key component of TeaNet is a reconstruction module that inputs randomly masked spectra and outputs reconstructed samples that are similar to the original ones, but include additional variations learned from the domain. These augmented samples are used to train the classification model. The reconstruction and prediction parts are trained simultaneously, end-to-end with back-propagation. Results on both synthetic and real-world datasets verified the superiority of the proposed method. In the most difficult synthetic scenarios TeaNet outperformed CNN by 17%. We visualized and analysed the neuron responses of TeaNet and CNN, and found that TeaNet's ability to identify discriminant wavenumbers was excellent compared to CNN. Our approach is general and can be easily adapted to other domains, offering a solution to more accurate and interpretable few-shot learning.

A number of studies have been developed recently in order to explore associations between craniofacial differences and genetics. Most of these works have been based on spatial face image models, adjusted for the counter effects of age. This approach provides a limited understanding of normal and abnormal craniofacial development owing to the lack of age progression information. Here, the authors propose and implement an imaging framework that combines facial landmark positioning, non‐rigid registration, novel age‐dependent face modelling and common distance metrics to disclose the most facial differences that vary across the time due to the subjects' age. All the experiments carried out and corresponding results presented here are based on a database comprising ordinary two‐dimensional (2D) frontal face images of Down Syndrome (DS) and control sample groups. A number of craniofacial metrics have been successfully identified that highlight statistically significant and clinically relevant differences between the controls and the faces associated with DS within the age range from 1 to 18 years old, producing realistic unbiased face models with similar level of detail at all age‐intervals, despite the small sample size available.

Recent advances in deep learning (DL) allow for solving complex AI problems that used to be considered very hard. While this progress has advanced many fields, it is considered to be bad news for Completely Automated Public Turing tests to tell Computers and Humans Apart (CAPTCHAs), the security of which rests on the hardness of some learning problems. In this paper, we introduce DeepCAPTCHA, a new and secure CAPTCHA scheme based on adversarial examples, an inherit limitation of the current DL networks. These adversarial examples are constructed inputs, either synthesized from scratch or computed by adding a small and specific perturbation called adversarial noise to correctly classified items, causing the targeted DL network to misclassify them. We show that plain adversarial noise is insufficient to achieve secure CAPTCHA schemes, which leads us to introduce immutable adversarial noise-an adversarial noise that is resistant to removal attempts. In this paper, we implement a proof of concept system, and its analysis shows that the scheme offers high security and good usability compared with the best previously existing CAPTCHAs.

The forensic analysis of digital images from mobile devices is particularly important given their quick expansion and everyday use in the society. A further consequence of digital images' widespread use is that they are used today as silent witnesses in legal proceedings, as crucial evidence of the crime. This study specifically addresses the description of a technique that allows the identification of the image source acquisition, for the specific case of mobile devices images. This approach is to extract wavelet‐based features from sensor pattern noise which are then classified using a support vector machine. Moreover, there are a number of parameters that allows the authors to adapt the execution of the algorithm to specific situations desired for the forensic analyst (a variety of types and sizes of image or optimising the average accuracy rate in terms of processing time). This article describes a set of experiments with the same set of images that can obtain general conclusions for the different configurations.

Facial composite systems are used to create a likeness to a suspect in criminal investigations. Traditional, feature-based facial composite systems rely on the witness' ability to recall individual features, provide verbal descriptions and then select them from stored libraries of labelled features - a task which witnesses often find difficult. The EFIT-V facial composite system is based on different principles, employing a holistic (whole face) approach to construction.

A facial composite system is described for use in criminal investigations which has distinct advantages over current methods. Unlike traditional feature based methods, our approach uses both local and global facial models, allowing a witness to evolve plausible, photo-realistic face images in an intuitive way. The basic method combines random sampling from a facial appearance model (AM) with an evolutionary algorithm (EA) to drive the search procedure to convergence. Three variants of the evolutionary algorithm have been explored and their performance measured using a computer simulation of a human witness (virtual witness). Further system functionality, provided by local appearance models and transformations of the appearance space which respectively allow both local features and semantic facial attributes to be manipulated, is presented. Preliminary examples of composites generated with our system are presented which demonstrate the potential superiority of the evolutionary approach to composite generation. 1