Biometric authentication systems must handle large and constantly evolving datasets and maintain high authentication accuracy and scalability. Existing probabilistic data structures like Bloom Filters (BFs) efficiently handle membership testing, but their conventional forms are inherently static and lack tolerance for noisy data. It is also an inherent challenge in biometric systems, where factors such as lighting or camera angle can introduce noise because of these variability. This paper proposes the Dynamic Hierarchical Bloom Filter (DHBF), a novel variant of the Bloom Filter designed to integrate noise tolerance and substring handling capabilities, while simultaneously offering enhanced scalability and dynamic adaptability. The DHBF accommodates fluctuating dataset sizes and mitigates the impact of noisy biometric samples on authentication accuracy by supporting flexible memory allocation. In this paper, experimental validation was performed on a dataset of 30,000 facial images. The results demonstrate that the DHBF achieves 100% authentication accuracy in dynamic operations such as enrollment, querying, insertion, and deletion. Additionally, our experiments demonstrate a reduction (≈ 30%) in storage requirements in terms of storing biometric templates compared to HBFs.

Two common attacks against biometric systems are direct (or physical) access and indirect (or logical) access. While most detection techniques focus on the former, often called presentation attacks, that occur at pre-sensor level, the attack surface for indirect access, that takes place post-sensor, is larger. In this paper, an indirect attack in the realm of faces is explored that utilizes a unique soft-biometric feature called ‘Kinship Cues’. Unlike gender and ethnicity, kinship is less explored but powerful; we find that its knowledge can significantly increase the chances of an attacker getting access to a system. Due to lack of kin data in other domains, our attack is only performed against facial biometric systems. Nevertheless, the results underscore the impact of kinship cues and their need to be investigated in other domains such as fingerprint and iris. This kinship artifact boosts the convergence speed of state-of-the-art iterative adaptive Bayesian hill climbing attacks. Further, it is exploited to generate a dictionary of input images, commonly called wolf images, in a novel kinship-based non-iterative indirect attack that we call Kin-Wolf. A classical image fusion technique (morphing) and a deep learning based kinship framework utilizing pre-trained StyleGAN2 are investigated to generate the wolf images. The trade-off between kinship cues and randomization is also studied and a 6× average improvement in attack accuracy is achieved for Kin-Wolf over random probes.

Advances in vision and deep learning have revolutionized feature extraction for face recognition and verification systems, yet, performing kinship verification from such features is still challenging. Ongoing research attempts to imitate a human by identifying features for kinship verification. In this paper, we propose KinfaceNet, a deep learning based kinship feature extractor, capable of extracting kinship features from a single input image independently without requiring its kin pair image. The base model of the method is adopted from face recognition domain which is then transfer learned in the domain of kinship by learning a distance mapping from face images to a compact Euclidean space where distances directly correspond to a measure of kinship similarity. Thus, unlike most of the works in deep learning based kinship domain, the extracted features can be used in many other applications such as image generation and family based clustering, etc. Training is performed by rearranging the data into classes of kin pairs and using a state-of-the-art triplet mining algorithm to address the unbalanced kinship data problem which causes overfitting. Also, one of the major advantages of our framework is that training can be performed on any face feature extractor model pre-trained on large face recognition data, thereby reducing training time by a considerable amount. Comparable verification accuracy is obtained from simple MLP network at only 20th epoch with KinfaceNet features extracted from the Family-In-the-Wild dataset, the largest in the wild kinship dataset available, as well as KinfaceW-I and II datasets.

Large-scale pre-training tasks like image classification, captioning, or self-supervised techniques do not incentivize learning the semantic boundaries of objects. However, recent generative foundation models built using text-based latent diffusion techniques may learn semantic boundaries. This is because they have to synthesize intricate details about all objects in an image based on a text description. Therefore, we present a technique for segmenting real and AI-generated images using latent diffusion models (LDMs) trained on internet-scale datasets. First, we show that the latent space of LDMs (z-space) is a better input representation compared to other feature representations like RGB images or CLIP encodings for text-based image segmentation. By training the segmentation models on the latent z-space, which creates a compressed representation across several domains like different forms of art, cartoons, illustrations, and photographs, we are also able to bridge the domain gap between real and AI-generated images. We show that the internal features of LDMs contain rich semantic information and present a technique in the form of LD-ZNet to further boost the performance of text-based segmentation. Overall, we show up to 6% improvement over standard baselines for text-to-image segmentation on natural images. For AI-generated imagery, we show close to 20% improvement compared to state-of-the-art techniques. The project is available at https://koutilya-pnvr.github.io/LD-ZNet/.

Fixed input graphs are a mainstay in approaches that utilize Graph Convolution Networks (GCNs) for knowledge transfer. The standard paradigm is to utilize relationships in the input graph to transfer information using GCNs from training to testing nodes in the graph; for example, the semi-supervised, zero-shot, and few-shot learning setups. We propose a generalized framework for learning and improving the input graph as part of the standard GCN-based learning setup. Moreover, we use additional constraints between similar and dissimilar neighbors for each node in the graph by applying triplet loss on the intermediate layer output. We present results of semi-supervised learning on Citeseer, Cora, and Pubmed benchmarking datasets, and zero/few-shot action recognition on UCF101 and HMDB51 datasets, significantly outperforming current approaches. We also present qualitative results visualizing the graph connections that our approach learns to update.

A Hierarchical Bloom Filter (HBF) -based biometric framework was recently proposed to provide compact storage, noise tolerance, and fast query processing for resource-constrained environments, e.g., Internet of things (IoT). While security and privacy were also touted as features of the HBF, it was not thoroughly evaluated. Compared to the classical BFs, the HBF uses a threshold parameter to make robust authentication decisions when the HBF encounters noise in the biometric input which one would think might lead to security issues. In this paper, the attack vectors that could compromise the HBF security by increasing the false positive authentication of non-members and by leaking soft information about enrolled members are explored. With quantitative analyses, HBF-based biometric system security under these well-defined attack vectors is evaluated and it is concluded that the framework is more difficult to attack than the classical Bloom Filter. Further, experimental results show that soft biometric information is also kept private.

Recent work has shown that the structure of convolutional neural networks (CNNs) induces a strong prior that favors natural images. This prior, known as a deep image prior (DIP), is an effective regularizer in inverse problems such as image denoising and inpainting. We extend the concept of the DIP to depth images. Given color images and noisy and incomplete target depth maps, we optimize a randomly-initialized CNN model to reconstruct a depth map restored by virtue of using the CNN network structure as a prior combined with a view-constrained photo-consistency loss. This loss is computed using images from a geometrically calibrated camera from nearby viewpoints. We apply this deep depth prior for inpainting and refining incomplete and noisy depth maps within both binocular and multi-view stereo pipelines. Our quantitative and qualitative evaluation shows that our refined depth maps are more accurate and complete, and after fusion, produces dense 3D models of higher quality.

We propose novel Stacked Spatio-Temporal Graph Convolutional Networks (Stacked-STGCN) for action segmentation, i.e., predicting and localizing a sequence of actions over long videos. We extend the Spatio-Temporal Graph Convolutional Network (STGCN) originally proposed for skeleton-based action recognition to enable nodes with different characteristics (e.g., scene, actor, object, action), feature descriptors with varied lengths, and arbitrary temporal edge connections to account for large graph deformation commonly associated with complex activities. We further introduce the stacked hourglass architecture to STGCN to leverage the advantages of an encoder-decoder design for improved generalization performance and localization accuracy. We explore various descriptors such as frame- level VGG, segment-level I3D, RCNN-based object, etc. as node descriptors to enable action segmentation based on joint inference over comprehensive contextual information. We show results on CAD120 (which provides pre-computed node features and edge weights for fair performance comparison across algorithms) as well as a more complex real- world activity dataset, Charades. Our Stacked-STGCN in general achieves improved performance over the state-of- the-art for both CAD120 and Charades. Moreover, due to its generic design, Stacked-STGCN can be applied to a wider range of applications that require structured inference over long sequences with heterogeneous data types and varied temporal extent.

Wide proliferation of counterfeit integrated circuits (ICs) is a major global threat. Currently, the process of counterfeit IC detection is time-consuming and requires highly skilled subject matter experts. In this paper, we have developed an automated image-processing-based methodology for recycled and remarked counterfeit IC detection, using images acquired through an ordinary optical microscope or a digital camera. The methodology has two phases: first, identification of counterfeit IC is attempted by package texture comparison of a golden IC sample and the given sample. Then, for the ICs which have not been inferred to be counterfeit, an optional second phase where detection of position and size of indents (or cavities) on the IC package surface is performed. Compared to previously proposed techniques, the proposed technique is less computationally expensive and avoids expensive equipment such as a scanning electron microscope or X-ray tomograph. Experimental results demonstrate that the proposed methodology achieves high detection accuracy, and the results are supported by an unsupervised clustering approach.

The widespread penetration of counterfeit integrated circuits (ICs) is not only a major threat to the electronic goods supply chain, but also constitute a great threat to national security. Image processing based counterfeit IC design techniques are promising, but currently often suffer from high computational complexity and requirement of expensive image acquisition infrastructure. We describe two techniques based on image texture analysis to automate the process of counterfeit IC detection. The first method employs local textural feature identification to detect counterfeit ICs. The second method includes identification of counterfeit ICs by segmenting the image into regions of different textural features using texture filters. The first method is of lower computational complexity compared to the segmentation method, but the second method is capable of blind identification in the sense that it does not require knowledge of the textural features of a golden IC sample. Our experimental results show that these methods have high detection accuracy, even for images acquired using ordinary digital cameras and low-end digital microscopes.