In the highly competitive landscape of e-commerce advertising, maximizing return on advertising spend (ROAS) is crucial yet inherently uncertain due to auction-based bidding dynamics and fluctuating market conditions. Traditional deterministic models struggle to capture this uncertainty, necessitating a probabilistic approach that balances predictive accuracy with interpretability. To address this challenge, the paper proposes a novel Hierarchical Bayesian Deep Learning framework. The architecture was motivated by initial exploratory analysis using a Bayesian Belief Network (BBN) to map structural dependencies, while the final deep learning model overcomes scalability limitations using self-attention mechanisms and a Mixture Density Network (MDN) for full distributional modeling of ROAS. The BBN captures dependencies among campaign variables, enhancing interpretability, while the hierarchical deep learning architecture leverages self-attention mechanisms to address scalability challenges in high-dimensional settings. Experimental results reveal that the proposed framework achieves 22.8% lower RMSE and 27.4% better Negative Log Likelihood (NLL) and up to 31.2% lower Kullback–Leibler divergence (KLD) than state-of-the-art methods (DeepAR, Prophet, NGBoost), achieving an R 2 of 98% with an inference speed of 5.2 ms per campaign, confirming its feasibility for real-time bidding applications which typically require sub-10ms latency, enabling a feasible real-time bidding. Ablation studies confirm that attention-driven feature selection and calibrated uncertainty quantification significantly enhance both predictive performance and explainability, identifying key drivers of campaign success. By providing precise, uncertainty-aware, and explainable predictions, this approach enables adaptive bidding strategies, optimized budget allocation, and risk management, setting a new benchmark for intelligent decision-making in digital advertising. • Proposed Hierarchical Bayesian Deep Learning model improves ROAS prediction accuracy. • Achieves 22.8% lower RMSE and 27.4% better Negative Log Likelihood than SOTA. • Combines Bayesian Belief and Mixture Density Networks for precise, uncertainty-aware predictions. • Enables real-time bidding with an inference speed of 5.2 ms per campaign. • Utilizes attention-driven feature selection for scalable and explainable predictions.

Passwords remain fundamental to user authentication, including handheld devices, wearables, personal computers, and network devices. Privacy concerns have led to the development of new password guidelines and alternatives, yet these have not seen widespread adoption among users. Increasing skepticism towards the service providers has made users reluctant to share sensitive information, including passwords. While current security protocols ensure data protection in transit, assurances regarding the security and privacy of data at rest are often assumed without verification. Traditional best practices for password storage involve hashing, which still requires the original password to be shared as plaintext or as a hash. Each of these methods has its vulnerabilities. For instance, an adversary can sniff network packets to capture the original password or the hash value, potentially compromising the authentication system. To address these issues, we propose a framework for password-based authentication using graph isomorphism as a zero-knowledge proof technique. This framework aims to replace conventional authentication methods and enhance password privacy. The results demonstrate the proposed framework's effectiveness in ensuring secure and private password authentication.

Reinforcement learning in the field of artificial intelligence has seen tremendous advances in recent years, but there are still several limitations standing in the way of its wider practical application, including sample inefficiency, generalization, and exploration-exploitation trade-off. Deep Q-Networks (DQN) have improved the performance of RL by using deep neural networks to approximate the Q-function and by using experience replay to store and reuse past experiences. Episodic memory in RL is a technique that allows an agent to store and reuse past experiences in order to improve its decision-making. However, current episodic memory-based RL techniques have some issues, such as generalization and slow learning, which can be improved by using methods such as experience replay compression and reducing the information of episodic memory into a parametric model. In this work, we propose cognitive-inspired episodic memory in DQN networks (CEMDQN) that reduces the priority weighting of old experiences over time, and the agent accesses recent experiences more frequently. The proposed model was evaluated on three different environments: StarGunner, BattleZone, and TimePilot. It was shown that when compared to standard episodic memory DQN, CEMDQN was more effective in test score performance for StarGunner (45.1 %), BattleZone (81 %), and TimePilot (53.54%) environments, respectively.

Drone-camera based human activity recognition (HAR) has received significant attention from the computer vision research community in the past few years. A robust and efficient HAR system has a pivotal role in fields like video surveillance, crowd behavior analysis, sports analysis, and human-computer interaction. What makes it challenging are the complex poses, understanding different viewpoints, and the environmental scenarios where the action is taking place. To address such complexities, in this paper, we propose a novel Sparse Weighted Temporal Attention (SWTA) module to utilize sparsely sampled video frames for obtaining global weighted temporal attention. The proposed SWTA is divided into two components. First, temporal segment network that sparsely samples a given set of frames. Second, weighted temporal attention, which incorporates a fusion of attention maps derived from optical flow, with raw RGB images. This is followed by a basenet network, which comprises a convolutional neural network (CNN) module along with fully connected layers that provide us with activity recognition. The SWTA network can be used as a plug-in module to the existing deep CNN architectures, for optimizing them to learn temporal information by eliminating the need for a separate temporal stream. It has been evaluated on three publicly available benchmark datasets, namely Okutama, MOD20, and Drone-Action. The proposed model has received an accuracy of 72.76%, 92.56%, and 78.86% on the respective datasets thereby surpassing the previous state-of-the-art performances by a margin of 25.26%, 18.56%, and 2.94%, respectively.

This paper investigates teeth-photo, a new biometric modality, for human authentication on mobile and hand-held devices. The proposed system is suitable for multiple applications including device unlocking and secure authentication. Teeth samples have been acquired using a mobile application having markers to register the teeth area. The region of interest (RoI) is then extracted using the markers and the same is enhanced for better visual clarity. A deep learning architecture along with the feature regularization scheme is devised to obtain highly discriminative embedding. The model is trained in an end-to-end manner with a few samples and thus, is efficient in terms of time and energy requirements. Experiments have been conducted on an in-house teeth-photo database collected using the proposed application from 92 subjects each providing 10 samples in multiple sessions over a span of 3–4 days. It has been observed that the proposed system achieved 97.61% accuracy with a Correct Recognition Rate (CRR) of 95% at an Equal Error Rate (EER) as low as 2.07% even for a small RoI of size 175 × 175. To the best of our understanding, this is the first work on teeth-photo-based authentication for mobile devices. The database along with the code is being made public.

Group Activity Recognition requires spatiotemporal modeling of an exponential number of semantic and geometric relations among various individuals in a scene. Previous attempts model these relations by aggregating independently derived spatial and temporal features. This increases the modeling complexity and results in sparse information due to lack of feature correlation. In this paper, we propose Weighted Temporal Attention Mechanism (WTM), a representational mechanism that combines spatial and temporal features of a local subset of a visual sequence into a single 2D image representation, highlighting areas of a frame where actor motion is significant. Pairwise dense optical flow maps representing the temporal characteristic of individuals over a sequence are used as attention masks over raw RGB images through a multi-layer weighted aggregation. We demonstrate a strong correlation between spatial and temporal features, which helps localize actions effectively in a multi-person scenario. The simplicity of the input representation allows the model to be trained by 2D image classification architectures in a plug-and-play fashion, which outperforms its multi-stream and multi-dimensional counterparts. The proposed method achieves the lowest computational complexity in comparison to other works. We demonstrate the performance of WTM on two widely used public benchmark datasets, namely the Collective Activity Dataset (CAD) and the Volleyball Dataset. and achieve state-of-the-art accuracies of 95.1% and 94.6% respectively. We also discuss the application of this method to other datasets and general scenarios. The code is being made publicly available.

Classifying complex human motion sequences is a major research challenge in the domain of human activity recognition. Currently, most popular datasets lack a specialized set of classes pertaining to similar action sequences (in terms of spatial trajectories). To recognize such complex action sequences with high inter-class similarity, such as those in karate, multiple streams are required. To fulfill this need, we propose MS-KARD, a Multi-Stream Karate Action Recognition Dataset that uses multiple vision perspectives, as well as sensor data - accelerometer and gyroscope. It includes 1518 video clips along with their corresponding sensor data. Each video was shot at 30fps and lasts around one minute, equating to a total of 2,814,930 frames and 5,623,734 sensor data samples. The dataset has been collected for 23 classes like Jodan Zuki, Oi Zuki, etc. The data acquisition setting involves the combination of 2 orthogonal web cameras and 3 wearable inertial sensors recording both vision and inertial data respectively. The aim of this dataset is to aid research that deals with recognizing human actions that have similar spatial trajectories. The paper describes statistics of the dataset, acquisition setting, and provides baseline performance figures using popular action recognizers. We propose an ensemble-based method, KarateNet, that performs decision-level fusion on the two input modalities (vision and sensor data) to classify actions. For the first stream, the RGB frames are extracted from the videos and passed into action recognition networks like Temporal Segment Network (TSN) and Temporal Shift Module (TSM). For the second stream, the sensor data is converted into a 2-D image and fed into a Convolutional Neural Network (CNN). The results reported were obtained on performing a fusion of the 2 streams. We also report results on ablations that use fusion with various input settings. The dataset and code will be made publicly available.

Human Activity Recognition is an important task in Computer Vision that involves the utilization of spatio-temporal features of videos to classify human actions. The temporal portion of videos contains vital information needed for accurate classification. However, common Deep Learning methods simply average the temporal features, thereby giving all frames equal importance irrespective of their relevance, which negatively impacts the accuracy of the model. To combat this adverse effect, this paper proposes a novel Transformer Based Attention Consensus (TBAC) module. The TBAC module can be used in a plug-and-play manner as an alternate to the conventional consensus meth-ods of any existing video action recognition network. The TBAC module contains four components: (i) Query Sampling Unit, (ii) Attention Extraction Unit, (iii) Softening Unit, and (iv) Attention Consensus Unit. Our experiments demonstrate that the use of the TBAC module in place of classical consensus can improve the performance of the CNN-based action recognition models, such as Channel Separated Convolutional Network (CSN), Temporal Shift Module (TSM), and Temporal Segment Network (TSN). We also propose the Decision Consensus (DC) algorithm that utilizes multiple independent but related action recognizer models in order to improve upon the performance of most of these constituent models, using a novel fusion algorithm. Results have been obtained on two benchmark human action recognition datasets, HMDB51 and HAA500. The use of the proposed TBAC module along with Decision Consensus achieves state-of-the-art performances, with 85.23% and 83.73% classification accuracies on the two databases HMDB51 and HAA500, respectively. The code will be made publicly available.

Yoga can be seen as a set of fitness exercises involving various body postures. Most of the available pose and action recognition datasets are comprised of easy-to-moderate body pose orientations and do not offer much challenge to the learning algorithms in terms of the complexity of pose. In order to observe action recognition from a different perspective, we introduce YogaTube, a new large-scale video benchmark dataset for yoga action recognition. YogaTube aims at covering a wide range of complex yoga postures, which consist of 5484 videos belonging to a taxonomy of 82 classes of yoga asanas. Also, a three-stream architecture has been designed for yoga asanas pose recognition using two modules, feature extraction, and classification. Feature extraction comprises three parallel components. First, pose is estimated using the part affinity fields model to extract meaningful cues from the practitioner. Second, optical flow is used to extract temporal features. Third, raw RGB videos are used for extracting the spatiotemporal features. Finally in the classification module, pose, optical flow, and RGB streams are fused to get the final results of the yoga asanas. To the best of our knowledge, this is the first attempt to establish a video benchmark yoga recognition dataset. The code and dataset will be released soon.

In the COVID-19 situation, face masks have become an essential part of our daily life. As mask occludes most prominent facial characteristics, it brings new challenges to the existing facial recognition systems. This paper presents an idea to consider forehead creases (under surprise facial expression) as a new biometric modality to authenticate mask-wearing faces. The forehead biometrics utilizes the creases and textural skin patterns appearing due to voluntary contraction of the forehead region as features. The proposed framework is an efficient and generalizable deep learning framework for forehead recognition. Face-selfie images are collected using smartphone’s frontal camera in an unconstrained environment with various indoor/outdoor realistic environments. Acquired forehead images are first subjected to a segmentation model that results in rectangular Region Of Interest (ROI’s). A set of convolutional feature maps are subsequently obtained using a backbone network. The primary embeddings are enriched using a dual attention network (DANet) to induce discriminative feature learning. The attention-empowered embeddings are then optimized using Large Margin Co-sine Loss (LMCL) followed by Focal Loss to update weights for inducting robust training and better feature discriminating capabilities. Our system is end-to-end and few-shot; thus, it is very efficient in memory requirements and recognition rate. Besides, we present a forehead image dataset (BITS-IITMandi-ForeheadCreases Images Database1) that has been recorded in two sessions from 247 subjects containing a total of 4,964 selfie-face mask images. To the best of our knowledge, this is the first to date mobile-based fore-head dataset and is being made available along with the mobile application in the public domain. The proposed system has achieved high performance results in both closed-set, i.e., CRR of 99.08% and EER of 0.44% and open-set matching, i.e., CRR: 97.84%, EER: 12.40% which justifies the significance of using forehead as a biometric modality.

Abstract Human activity recognition aims to determine actions performed by a human in an image or video. Examples of human activity include standing, running, sitting, sleeping,etc. These activities may involve intricate motion patterns and undesired events such as falling. This paper proposes a novel deep convolutional long short-term memory (ConvLSTM) network for skeletal-based activity recognition and fall detection. The proposed ConvLSTM network is a sequential fusion of convolutional neural networks (CNNs), long short-term memory (LSTM) networks, and fully connected layers. The acquisition system applies human detection and pose estimation to pre-calculate skeleton coordinates from the image/video sequence. The ConvLSTM model uses the raw skeleton coordinates along with their characteristic geometrical and kinematic features to construct the novel guided features. The geometrical and kinematic features are built upon raw skeleton coordinates using relative joint position values, differences between joints, spherical joint angles between selected joints, and their angular velocities. The novel spatiotemporal-guided features are obtained using a trained multi-player CNN-LSTM combination. Classification head including fully connected layers is subsequently applied. The proposed model has been evaluated on the KinectHAR dataset having 130,000 samples with 81 attribute values, collected with the help of a Kinect (v2) sensor. Experimental results are compared against the performance of isolated CNNs and LSTM networks. Proposed ConvLSTM have achieved an accuracy of 98.89% that is better than CNNs and LSTMs having an accuracy of 93.89 and 92.75%, respectively. The proposed system has been tested in realtime and is found to be independent of the pose, facing of the camera, individuals, clothing,etc. The code and dataset will be made publicly available.

Recently, deep hierarchically learned models (such as CNN) have achieved superior performance in various computer vision tasks but limited attention has been paid to biometrics till now. This is major because of the number of samples available in biometrics are limited and are not enough to train CNN efficiently. However, deep learning often requires a lot of training data because of the huge number of parameters to be tuned by the learning algorithm. How about designing an end-to-end deep learning network to match the biometric features when the number of training samples is limited? To address this problem, we propose a new way to design an end-to-end deep neural network that works in two major steps: first an auto-encoder has been trained for learning domain specific features followed by a Siamese network trained via. triplet loss function for matching. A publicly available vein image data set has been utilized as a case study to justify our proposal. We observed that transformations learned from such a network provide domain specific and most discriminative vascular features. Subsequently, the corresponding traits are matched using multimodal pipelined end-to-end network in which the convolutional layers are pre-trained in an unsupervised fashion as an autoencoder. Thorough experimental studies suggest that the proposed framework consistently outperforms several state-of-the-art vein recognition approaches.

The aforementioned works and other analogous studies in finger knuckle images recognition have claimed that the precise detection of true features is difficult from poorly segmented images and the main reason for matching errors. Thus, an accurate segmentation of the region of interest is very crucial to achieve superior recognition results. In this paper, we have proposed a novel holistic and generalized segmentation Network (HFDSegNet) that automatically categorizes the given finger dorsal image obtained from multiple sensory resources into particular class and then extracts three possible ROIs (major knuckle, minor knuckle and nail) accurately. To best of our knowledge, this is the first attempt, an end-to-end trained object detector inspired by Deep Learning technique namely faster R-CNN (Region based Convolutional Neural Network) has been employed to detect and localize the position of finger knuckles and nail, even finger images exhibit blur, occlusion, low contrast etc. The experimental results are examined on two publicly available databases named as Poly-U contact-less FKI data-set, and Poly U FKP database. The proposed network is trained only over 500 randomly selected images per database, demonstrate the outstanding performance of proposed ROI’s segmentation network.

Large scale adaptation of a fingerprint based recognition system results in expansion of its database which increases the cost of identification and degrades the system performance. The paper proposes an efficient indexing technique that can scatter the effect of database escalation and maintains the system performance. It has proposed a fixed length feature vector built from each minutia, known as Coaxial Gaussian Track Code (CGTC). The proposed technique inserts feature vector into a Quantized Lookup Table (QLT) only once. As a result, it reduces both computational and memory costs. Since minutiae of all fingerprint images in the database are found to be well distributed in the quantized lookup table, it does not need rehashing. Experiments have been conducted over three fingerprint databases viz. FVC2002, FVC2004 and IITK-Sel500FP containing fingerprints from 100, 100 and 500 subjects respectively. Results have proven the superiority of the proposed indexing technique against well known geometric based indexing techniques.

This paper proposes a transaction strategy to implement transferable E-cash. It does not involve any third party observer at the time of transaction and possesses strong anonymity property. It uses biometric features of the legitimate holder of e-cash to progress the transaction. It works on the transfer of coin ownership and is built around the restrictive blinding and biometric digital signature scheme. Its mathematical model utilizes the hardness of discrete log problem and representation problem in groups. It guarantees anonymity and unlinkability for a genuine user and non repudiation to a fraudulent one.