Saliency-based explainable AI (XAI) methods are commonly used to explain the behaviors of AI models, despite the limited research on whether such methods can indeed enhance user understanding. Here we proposed a set of tasks to systematically and objectively evaluate user’s global understanding of object detection models at the feature, object, and image levels. We found that while presenting AI’s hits, misses, and false alarms to users could enhance feature-level and some aspects of object-level understanding, presenting saliency-based explanations could not provide any additional help and did not help direct user’s attention to relevant features. Meanwhile, presenting AI’s hits, misses, and false alarms alone did not help users distinguish AI’s hits from misses and did not enhance image-level understanding. At the image level, among the participants, assuming that AI would behave like themselves appeared to be the best strategy for predicting AI’s behavior, since any attempts to revise such assumption resulted in further deviations from AI’s actual behaviors. Thus, it is necessary to develop more effective XAI methods, particularly for object detection models. Our eye movement analyses showed that participants who used similar strategies to AI also tended to perform more similarly to AI, suggesting that we could instruct users to use their own strategy as a reference point to predict AI’s behavior accordingly. Also, participants’ eye movement consistency and attention strategy similarity to AI’s were associated with different aspects of user understanding, suggesting that eye movements could be used as non-intrusive measures to monitor user understanding for providing user-specific explanations in future XAI methods.

Many students enjoy listening to background music (BGM) when they read, but it is challenging to measure their meta-cognitive states (e.g., understanding of the passage, engagement in reading). Eye movements, as an approach in multimodal learning analytics (MmLA), can offer continuous fine-grained data that reflect learners’ cognitive processes. This study explored the potential of utilizing eye movement measures to predict learners’ meta-cognition during reading with BGM. Results showed that learners’ eye movement measures integrated with the characteristics of the BGM, learner traits, and text complexity could predict their meta-cognitive states in reading. Findings can advance our understanding of human meta-cognition in multi-channel learning settings and provide insights for personalized BGM recommendations to enhance reading experiences.

Explainable artificial intelligence (XAI) has been increasingly investigated to enhance the transparency of black-box artificial intelligence models, promoting better user understanding and trust. Developing an XAI that is faithful to models and plausible to users is both a necessity and a challenge. This work examines whether embedding human attention knowledge into saliency-based XAI methods for computer vision models could enhance their plausibility and faithfulness. Two novel XAI methods for object detection models, namely FullGrad-CAM and FullGrad-CAM++, were first developed to generate object-specific explanations by extending the current gradient-based XAI methods for image classification models. Using human attention as the objective plausibility measure, these methods achieve higher explanation plausibility. Interestingly, all current XAI methods when applied to object detection models generally produce saliency maps that are less faithful to the model than human attention maps from the same object detection task. Accordingly, human attention-guided XAI (HAG-XAI) was proposed to learn from human attention how to best combine explanatory information from the models to enhance explanation plausibility by using trainable activation functions and smoothing kernels to maximize the similarity between XAI saliency map and human attention map. The proposed XAI methods were evaluated on widely used BDD-100K, MS-COCO, and ImageNet datasets and compared with typical gradient-based and perturbation-based XAI methods. Results suggest that HAG-XAI enhanced explanation plausibility and user trust at the expense of faithfulness for image classification models, and it enhanced plausibility, faithfulness, and user trust simultaneously and outperformed existing state-of-the-art XAI methods for object detection models.

We propose the gradient-weighted Object Detector Activation Maps (ODAM), a visual explanation technique for interpreting the predictions of object detectors. Utilizing the gradients of detector targets flowing into the intermediate feature maps, ODAM produces heat maps that show the influence of regions on the detector's decision for each predicted attribute. Compared to previous works on classification activation maps (CAM), ODAM generates instance-specific explanations rather than class-specific ones. We show that ODAM is applicable to one-stage, two-stage, and transformer-based detectors with different types of detector backbones and heads, and produces higher-quality visual explanations than the state-of-the-art in terms of both effectiveness and efficiency. We discuss two explanation tasks for object detection: 1) object specification: what is the important region for the prediction? 2) object discrimination: which object is detected? Aiming at these two aspects, we present a detailed analysis of the visual explanations of detectors and carry out extensive experiments to validate the effectiveness of the proposed ODAM. Furthermore, we investigate user trust on the explanation maps, how well the visual explanations of object detectors agrees with human explanations, as measured through human eye gaze, and whether this agreement is related with user trust. Finally, we also propose two applications, ODAM-KD and ODAM-NMS, based on these two abilities of ODAM. ODAM-KD utilizes the object specification of ODAM to generate top-down attention for key predictions and instruct the knowledge distillation of object detection. ODAM-NMS considers the location of the model's explanation for each prediction to distinguish the duplicate detected objects. A training scheme, ODAM-Train, is proposed to improve the quality on object discrimination, and help with ODAM-NMS.

The face recognition of familiar and unfamiliar people is an essential part of our daily lives. However, its neural mechanism and relevant electroencephalography (EEG) features are still unclear. In this study, a new EEG-based familiar and unfamiliar faces classification method is proposed. We record the multichannel EEG with three different face-recall paradigms, and these EEG signals are temporally segmented and filtered using a well-designed filter-bank strategy. The filter-bank differential entropy is employed to extract discriminative features. Finally, the support vector machine (SVM) with Gaussian kernels serves as the robust classifier for EEG-based face recognition. In addition, the F-score is employed for feature ranking and selection, which helps to visualize the brain activation in time, frequency, and spatial domains, and contributes to revealing the neural mechanism of face recognition. With feature selection, the highest mean accuracy of 74.10% can be yielded in face-recall paradigms over ten subjects. Meanwhile, the analysis of results indicates that the EEG-based classification performance of face recognition will be significantly affected when subjects lie. The time–frequency topographical maps generated according to feature importance suggest that the delta band in the prefrontal region correlates to the face recognition task, and the brain response pattern varies from person to person. The present work demonstrates the feasibility of developing an efficient and interpretable brain–computer interface for EEG-based face recognition.

The hidden Markov model (HMM) is a broadly applied generative model for representing time-series data, and clustering HMMs attract increased interest from machine learning researchers. However, the number of clusters ( K ) and the number of hidden states ( S ) for cluster centers are still difficult to determine. In this article, we propose a novel HMM-based clustering algorithm, the variational Bayesian hierarchical EM algorithm, which clusters HMMs through their densities and priors and simultaneously learns posteriors for the novel HMM cluster centers that compactly represent the structure of each cluster. The numbers K and S are automatically determined in two ways. First, we place a prior on the pair (K,S) and approximate their posterior probabilities, from which the values with the maximum posterior are selected. Second, some clusters and states are pruned out implicitly when no data samples are assigned to them, thereby leading to automatic selection of the model complexity. Experiments on synthetic and real data demonstrate that our algorithm performs better than using model selection techniques with maximum likelihood estimation.

Model explanations are generated by XAI (explainable AI) methods to help people understand and interpret machine learning models. To study XAI methods from the human perspective, we propose a human-based benchmark dataset, i.e., human saliency benchmark (HSB), for evaluating saliency-based XAI methods. Different from existing human saliency annotations where class-related features are manually and subjectively labeled, this benchmark collects more objective human attention on vision information with a precise eye-tracking device and a novel crowdsourcing experiment. Taking the labor cost of human experiment into consideration, we further explore the potential of utilizing a prediction model trained on HSB to mimic saliency annotating by humans. Hence, a dense prediction problem is formulated, and we propose an encoder-decoder architecture which combines multi-modal and multi-scale features to produce the human saliency maps. Accordingly, a pretraining-finetuning method is designed to address the model training problem. Finally, we arrive at a model trained on HSB named human saliency imitator (HSI). We show, through an extensive evaluation, that HSI can successfully predict human saliency on our HSB dataset, and the HSI-generated human saliency dataset on ImageNet showcases the ability of benchmarking XAI methods both qualitatively and quantitatively.

Reading is an essential medium for learning, but it is challenging to measure learners’ cognitive processes during reading. Eye-tracking, as an approach in multimodal learning analytics (MmLA), can provide fine-grained data that reflect cognitive processes during reading. In this study, we investigated whether eye movements could predict passage reading performance in addition to language proficiency and cognitive abilities. In particular, we assessed learners’ eye movement pattern and consistency through a novel method, Eye Movement analysis with Hidden Markov Models (EMHMM), in addition to traditional eye movement measures. We found that longer saccade length predicted faster reading speed Also, higher English proficiency predicted faster reading speed through the mediation of longer saccade length. In contrast, reading comprehension accuracy was best predicted by a more consistent eye fixation at the beginning of reading engagement, which may result from a better developed visual routine due to higher reading expertise. These findings have important implications for ways to assess and facilitate learners’ reading through eye movement measures and to examine factors influencing reading performance. The methods adopted could further the development of MmLA and serve as an empirical example of understanding learners’ cognitive processes through collecting and modeling critical learner-centered metrics in novel modalities.

Perturbation-based techniques are promising for explaining black-box machine learning models due to their effectiveness and ease of implementation. However, prior works have faced the problem of Out-of-Distribution (OoD) — an artifact of randomly perturbed data becoming inconsistent with the original dataset, degrading the reliability of generated explanations, which is still under-explored according to our best knowledge. This work addresses the OoD issue by designing a simple yet effective module that can quantify the affinity between the perturbed data and the original dataset distribution. Specifically, we penalize the influences of unreliable OoD data for the perturbed samples by integrating the inlier scores and prediction results of the target models, thereby making the final explanations more robust. Our solution is shown to be compatible with the most popular perturbation-based XAI algorithms: RISE, OCCLUSION, and LIME. Extensive experiments confirmed that our methods exhibit superior performance in most cases with computational and cognitive metrics. In particular, we point out the degradation problem of RISE algorithm for the first time. With our design, the performance of RISE can be boosted significantly. Besides, our solution also resolves a fundamental problem with a faithfulness indicator, a commonly used evaluation metric of XAI algorithms that appears sensitive to the OoD issue.

We propose a ParametRIc MAnifold Learning (PRIMAL) algorithm for Gaussian mixtures models (GMM), assuming that GMMs lie on or near to a manifold of probability distributions that is generated from a low-dimensional hierarchical latent space through parametric mappings. Inspired by principal component analysis (PCA), the generative processes for priors, means and covariance matrices are modeled by their respective latent space and parametric mapping. Then, the dependencies between latent spaces are captured by a hierarchical latent space by a linear or kernelized mapping. The function parameters and hierarchical latent space are learned by minimizing the reconstruction error between ground-truth GMMs and manifold-generated GMMs, measured by Kullback-Leibler Divergence (KLD). Variational approximation is employed to handle the intractable KLD between GMMs and a variational EM algorithm is derived to optimize the objective function. Experiments on synthetic data, flow cytometry analysis, eye-fixation analysis and topic models show that PRIMAL learns a continuous and interpretable manifold of GMM distributions and achieves a minimum reconstruction error.

The Gaussian Mixture Model (GMM) is among the most widely used parametric probability distributions for representing data. However, it is complicated to analyze the relationship among GMMs since they lie on a high-dimensional manifold. Previous works either perform clustering of GMMs, which learns a limited discrete latent representation, or kernel-based embedding of GMMs, which is not interpretable due to difficulty in computing the inverse mapping. In this paper, we propose Parametric Manifold Learning of GMMs (PML-GMM), which learns a parametric mapping from a low-dimensional latent space to a high-dimensional GMM manifold. Similar to PCA, the proposed mapping is parameterized by the principal axes for the component weights, means, and covariances, which are optimized to minimize the reconstruction loss measured using Kullback-Leibler divergence (KLD). As the KLD between two GMMs is intractable, we approximate the objective function by a variational upper bound, which is optimized by an EM-style algorithm. Moreover, We derive an efficient solver by alternating optimization of subproblems and exploit Monte Carlo sampling to escape from local minima. We demonstrate the effectiveness of PML-GMM through experiments on synthetic, eye-fixation, flow cytometry, and social check-in data.

Since Yarbus's seminal work in 1965, vision scientists have argued that people's eye movement patterns differ depending upon their task. This suggests that we may be able to infer a person's task (or mental state) from their eye movements alone. Recently, this was attempted by Greene et al. [2012] in a Yarbus-like replication study; however, they were unable to successfully predict the task given to their observer. We reanalyze their data, and show that by using more powerful algorithms it is possible to predict the observer's task. We also used our algorithms to infer the image being viewed by an observer and their identity. More generally, we show how off-the-shelf algorithms from machine learning can be used to make inferences from an observer's eye movements, using an approach we call Multi-Fixation Pattern Analysis (MFPA).