Online education has gained significant popularity during the COVID-19 pandemic. The availability of massive open online courses has further strengthened the online learning environment. One of the considerable challenges in online learning environments is to measure the learner’s engagement to meet the educational objectives. This paper addresses the challenge and proposes a convolutional neural network-based architecture that extracts discriminative features from the face (Affective) and the upper body (Behavioral) of the learner and classifies the engagement. The performance of the proposed architecture is evaluated on the publicly available Dataset of Affective States In E-Environments (DAiSEE) and the learning-centered affective state dataset curated from open-source datasets. The experimental results demonstrate that the proposed methodology with affective and behavioral features improves the engagement measurement results. The proposed method outperforms the state-of-the-art in terms of Unweighted Average Recall (UAR), Unweighted Average Precision (UAP), and Unweighted Average F1 (UAF1) with 79.14%, 49.62%, and 55.29% values, respectively. It is threefold more computationally efficient than the previous state-of-the-art method. The proposed method also improves the accuracy of the curated dataset by 2.07% of the prior method.

Availability of the internet and quality of content attracted more learners to online platforms that are stimulated by COVID-19. Students of different cognitive capabilities join the learning process. However, it is challenging for the instructor to identify the level of comprehension of the individual learner, specifically when they waver in responding to feedback. The learner's facial expressions relate to content comprehension and engagement. This paper presents use of the vision transformer (ViT) to model automatic estimation of student engagement by learning the end-to-end features from facial images. The ViT architecture is used to enlarge the receptive field of the architecture by exploiting the multi-head attention operations. The model is trained using various loss functions to handle class imbalance. The ViT is evaluated on Dataset for Affective States in E-Environments (DAiSEE); it outperformed frame level baseline result by approximately 8% and the other two video level benchmarks by 8.78% and 2.78% achieving an overall accuracy of 55.18%. In addition, ViT with focal loss was also able to produce well distribution among classes except for one minority class.

Abstract This paper proposes a novel algorithm for computing discriminative descriptors named as a sparse coded composite descriptor (SCCD) for robust human activity recognition. The proposed method blends the state‐of‐the‐art handcrafted features and the discriminative nature of the sparse representation of visual information. The human activity is firstly modelled using any handcrafted feature, and then the sparse codes computed on a discriminative sparse dictionary of these features are embedded to provide discrimination in the feature set. Finally, a support vector machine (SVM) is trained using the proposed SCCDs to perform classification of different human activities. A new feature named as differential motion descriptor (DMD) is also proposed to extract the motion as well as spatial information from an activity video. The simulation results reveal that in comparison with the handcrafted feature, the corresponding SCCD improves the recognition accuracy significantly. The proposed method is compared with state‐of‐the‐art methods on KTH, Ballet, UCF50, and HMDB51 datasets and the proposed methodology of composite features outperforms these methods in terms of recognition accuracy.

Genetic interactions are very helpful in understanding different disease and discovering drugs for it. Compared to the gene pairs that represent the genetic interactions between two genes, the gene triplets are more informative and useful. However, existing works on genetic interactions among gene triplets have primarily focused on detecting gene triplets from time series gene expression profiles. Generating the time series gene expression profiles for humans is quite impracticable but the labeled gene expression profiles are available for different diseases in case of humans. In this paper, a computational framework has been proposed to detect gene triplets from labeled gene expression profiles. First, it employs Rough Set Theory for extracting the key genes and then designs a fuzzy inference system for generating possible gene triplets. Further, Root Mean Squared Error measure has been used to prune out the irrelevant gene triplets. In the present work, the proposed computational framework has been applied to labeled lung adenocarcinoma dataset and can be applied to any other labeled gene expression dataset. The extracted gene triplets and their functionalities have been verified with existing biological literature and benchmark databases and the results of verification signify that the proposed framework is promising in terms of finding useful genetic triplets. Further, the proposed framework has been found more efficient as compared to an existing mutual information-based technique in terms of detecting known genetic interactions.

Multimodal biometric systems provide prospective advantages over traditional unimodal systems and are employed diversely for numerous applications. However, issues of data privacy and identity-theft emerges with the extensive use of these systems. To address these issues we propose a multimodal cancelable biometric system based on realtime Deep Feature Unification (DFU). For this, keys-based generic feature extraction is introduced to achieve revocability and dimensionality reduction. Non-invertibility is obtained through random projection of Key Deep features to Query Deep features. Proposed adaptive graph-based fusion process not only extracts complementary information across multiple modalities but also generates multimodal Unified template. The cross diffusion of normalized and optimal graphs ensure the unlinkability and robustness to dynamic environment. Proposed biometric system is assessed over benchmark datasets and shows promising performance against state-of-the-art methods. Average DI and EER achieved by proposed method are 10.35 and 0.12, respectively. Further, robustness against adversary attacks is demonstrated.

Multimodal biometric systems offer numerous advantages over unimodal counterparts and are being used extensively in diverse applications. However, fusion of biometric data is a non-trivial task and curtail employability of multimodal systems for a varying set of biometric characteristics with different type and dimension. Moreover, comprehensive solutions against adversary attacks that ensure template protection and prevent presentation attacks are not in place. In this article, a secure multimodal cancelable biometric system is proposed to address these concerns. This approach introduces key images based generic feature extraction technique which reduces feature dimension and achieves revocability. The non-invertibility and unlinkability are ensured through cross-diffusion of complementary information from different modalities. A new feature fusion method based on an adaptive graph is proposed to generate multimodal cancelable biometric templates. Robustness against presentation attack is accomplished through quality based adaptation of features. Extensive experimentation is performed on benchmark databases for fingerprint, face, and iris, to illustrate the efficacy of multimodal cancelable templates. The proposed approach is shown to perform favorably against state-of-the-art feature fusion methods. Furthermore, the resilience of the proposed approach against security and privacy attacks is demonstrated.

In this paper, a novel approach is employed for fingerprint image super-resolution based on sparse representation over a set of coupled low and high-resolution dictionary pairs. The primary step of fingerprint super-resolution involves learning a pair of coupled low- and high-resolution sub-dictionaries for each cluster of patches sampled from training set of fingerprint images. The clusters are formulated based on patch sharpness and the dominant phase angle via the magnitude and phase of the gradient operator for each image patch. In the reconstruction stage, for the low-resolution patch the most appropriate dictionary pair is selected, and the sparse coding coefficients are calculated with respect to the low-resolution dictionary. The equality assumption of the sparse representation of the low and high-resolution patches is the link between the low and high-resolution features space. For the reconstruction of high resolution patch, the sparse coefficients calculated for low-resolution patch are directly multiplied with corresponding high-resolution dictionary. The conducted experiments over fingerprint images show that the algorithm is competitive with the state-of-art super-resolution algorithms.