Tactile graphics enable individuals with visual impairment to interpret visual information through touch, supporting navigation, education, and social engagement. However, manually designing tactile graphics is costly, labor-intensive, and difficult to scale. This work introduces a text-guided image-to-image translation approach to generate tactile maps from RGB maps. By leveraging natural language prompts, the method allows control over map elements such as lakes, rivers, and cities, enabling customization based on specific needs. To train the model, we created a custom dataset consisting of 1,845 RGB maps of Canadian provinces, each paired with multiple tactile variations reflecting different levels of detail. Corresponding text prompts were designed to describe these variations, forming a dataset of 9,800 triplets (RGB map, tactile map, prompt). Human expert assessments demonstrated that the proposed method outperforms a baseline model, with 47% of the outputs requiring minimal adjustments. The results highlight a scalable and efficient solution for tactile map generation, ensuring high-quality outputs while maintaining adaptability through text-based control.

Tactile graphics are essential for providing access to visual information for the 43 million people globally living with vision loss. Traditional methods for creating these graphics are labor-intensive and cannot meet growing demand. We introduce TactileNet, the first comprehensive dataset and AI-driven framework for generating embossing-ready 2D tactile templates using text-to-image Stable Diffusion models. We fine-tune Stable Diffusion models using Low-Rank Adaptation and DreamBooth to generate high-fidelity, guideline-compliant graphics with reduced computational cost. Quantitative evaluations with tactile experts show 92.86% adherence to accessibility standards. Our structural fidelity analysis revealed near-human design similarity, with a Structural Similarity Index (SSIM) of 0.538 between generated and expert-designed tactile images. Notably, our method better preserves object silhouettes than human designs (binary mask SSIM: 0.259 vs. 0.215), addressing a key limitation of manual abstraction. The framework scales to 32,000 images (7,050 high-quality) across 66 classes, with prompt editing enabling customizable outputs (e.g., adding or removing details). By automating the 2D template generation step compatible with standard embossing workflows—TactileNet accelerates production while preserving design flexibility. This work demonstrates how AI can augment (not replace) human expertise to bridge the accessibility gap in education and beyond. Code, data, and models can be found at our project page https://tactilenet.github.io/.

Zero-shot visual question answering (VQA) poses a formidable challenge at the intersection of computer vision and natural language processing. Traditionally, this problem has been tackled using end-to-end pre-trained vision-language models (VLMs). However, recent advancements in large language models (LLMs) demonstrate their exceptional reasoning and comprehension abilities, making them valuable assets in multi-modal tasks, including zero-shot VQA. LLMs have been previously integrated with VLMs to solve zero-shot VQA in a conversation-based approach. However, while the focus in VQA tasks is often on specific regions rather than the entire image, this aspect has been overlooked in previous approaches. Consequently, the overall performance of the framework relies on the ability of the pre-trained VLM to locate the region of interest that is relevant to the requested visual information within the entire image. To address this challenge, this paper proposes Grounded Multi-modal Conversation for Zero-shot Visual Question Answering (GMC-VQA), a region-based framework that leverages the complementary strengths of LLMs and VLMs in a conversation-based approach. We employ a grounding mechanism to refine visual focus according to the semantics of the question and foster collaborative interaction between VLM and LLM, effectively bridging the gap between visual and textual modalities and enhancing comprehension and response generation for visual queries. We evaluate GMC-VQA across three diverse VQA datasets, achieving substantial average improvements of 10.04% over end-to-end VLMs and 2.52% over the state-of-the-art VLM-LLM communication-based framework, respectively. Our code is publicly available at https://github.com/mrzarei5/GMC-VQA.

Vaccine hesitancy continues to be a main challenge for public health officials during the COVID-19 pandemic. As this hesitancy undermines vaccine campaigns, many researchers have sought to identify its root causes, finding that the increasing volume of anti-vaccine misinformation on social media platforms is a key element of this problem. We explored Twitter as a source of misleading content with the goal of extracting overlapping cultural and political beliefs that motivate the spread of vaccine misinformation. To do this, we have collected a data set of vaccine-related Tweets and annotated them with the help of a team of annotators with a background in communications and journalism. Ultimately we hope this can lead to effective and targeted public health communication strategies for reaching individuals with anti-vaccine beliefs. Moreover, this information helps with developing Machine Learning models to automatically detect vaccine misinformation posts and combat their negative impacts. In this paper, we present Vax-Culture, a novel Twitter COVID-19 dataset consisting of 6373 vaccine-related tweets accompanied by an extensive set of human-provided annotations including vaccine-hesitancy stance, indication of any misinformation in tweets, the entities criticized and supported in each tweet and the communicated message of each tweet. Moreover, we define five baseline tasks including four classification and one sequence generation tasks, and report the results of a set of recent transformer-based models for them. The dataset and code are publicly available at https://github.com/mrzarei5/Vax-Culture.

Few-shot learning aims at recognizing new instances from classes with limited samples. This challenging task is usually alleviated by performing meta-learning on similar tasks. However, the resulting models are black-boxes. There has been growing concerns about deploying black-box machine learning models and FSL is not an exception in this regard. In this paper, we propose a method for FSL based on a set of human-interpretable concepts. It constructs a set of metric spaces associated with the concepts and classifies samples of novel classes by aggregating concept-specific decisions. The proposed method does not require concept annotations for query samples. This interpretable method achieved results on a par with six previously state-of-the-art black-box FSL methods on the CUB fine-grained bird classification dataset.

We propose a differentiable loss function for learning an embedding space by minimizing the upper bound of the leave-one-out classification error rate of 1-nearest neighbor classification error in the latent space. To evaluate the resulting space, in addition to the classification performance, we examine the problem of finding subclasses. In many applications, it is desired to detect unknown subclasses that might exist within known classes. For example, discovering subtypes of a known disease may help develop customized treatments. Analogous to the hierarchical clustering, subclasses might exist on different scales. The proposed method provides a mechanism to target subclasses in different scales.

Growing concerns regarding the operational usage of AI models in the real-world has caused a surge of interest in explaining AI models’ decisions to humans. Reinforcement Learning is not an exception in this regard. In this work, we propose a method for offering local explanations on risk in reinforcement learning. Our method only requires a log of previous interactions between the agent and the environment to create a state-transition model. It is designed to work on RL environments with either continuous or discrete state and action spaces. After creating the model, actions of any agent can be explained in terms of the features most influential in increasing or decreasing risk or any other desirable objective function in the locality of the agent. Through experiments, we demonstrate the effectiveness of the proposed method in providing such explanations.

In many scenarios, human decisions are explained based on some high-level concepts. In this work, we take a step in the interpretability of neural networks by examining their internal representation or neuron’s activations against concepts. A concept is characterized by a set of samples that have specific features in common. We propose a framework to check the existence of a causal relationship between a concept (or its negation) and task classes. While the previous methods focus on the importance of a concept to a task class, we go further and introduce four measures to quantitatively determine the order of causality. Through experiments, we demonstrate the effectiveness of the proposed method in explaining the relationship between a concept and the predictive behaviour of a neural network.

In many real-world scenarios, data from multiple modalities (sources) are collected during a development phase. Such data are referred to as multiview data. While additional information from multiple views often improves the performance, collecting data from such additional views during the testing phase may not be desired due to the high costs associated with measuring such views or, unavailability of such additional views. Therefore, in many applications, despite having a multiview training data set, it is desired to do performance testing using data from only one view. In this paper, we present a multiview feature selection method that leverages the knowledge of all views and use it to guide the feature selection process in an individual view. We realize this via a multiview feature weighting scheme such that the local margins of samples in each view are maximized and similarities of samples to some reference points in different views are preserved. Also, the proposed formulation can be used for cross-view matching when the view-specific feature weights are pre-computed on an auxiliary data set. Promising results have been achieved on nine real-world data sets as well as three biometric recognition applications. On average, the proposed feature selection method has improved the classification error rate by 31 percent of the error rate of the state-of-the-art.

Mismatch negativity (MMN) is a component of the event-related potential (ERP) that is elicited through an odd-ball paradigm. The existence of the MMN in a coma patient has a good correlation with coma emergence; however, this component can be difficult to detect. Previously, MMN detection was based on visual inspection of the averaged ERPs by a skilled clinician, a process that is expensive and not always feasible in practice. In this paper, we propose a practical machine learning (ML) based approach for detection of MMN component, thus, improving the accuracy of prediction of emergence from coma. Furthermore, the method can operate on an automatic and continuous basis thus alleviating the need for clinician involvement. The proposed method is capable of the MMN detection over intervals as short as two minutes. This finer time resolution enables identification of waxing and waning cycles of a conscious state. An auditory odd-ball paradigm was applied to 22 healthy subjects and 2 coma patients. A coma patient is tested by measuring the similarity of the patient's ERP responses with the aggregate healthy responses. Because the training process for measuring similarity requires only healthy subjects, the complexity and practicality of training procedure of the proposed method are greatly improved relative to training on coma patients directly. Since there are only two coma patients involved with this study, the results are reported on a very preliminary basis. Preliminary results indicate we can detect the MMN component with an accuracy of 92.7% on healthy subjects. The method successfully predicted emergence in both coma patients when conventional methods failed. The proposed method for collecting training data using exclusively healthy subjects is a novel approach that may prove useful in future, unrelated studies where ML methods are used.

Electrocardiogram (ECG) and transient evoked otoacoustic emission (TEOAE) are among the physiological signals that have attracted significant interest in biometric community due to their inherent robustness to replay and falsification attacks. However, they are time-dependent signals and this makes them hard to deal with in across-session human recognition scenario where only one session is available for enrollment. This paper presents a novel feature selection method to address this issue. It is based on an auxiliary dataset with multiple sessions where it selects a subset of features that are more persistent across different sessions. It uses local information in terms of sample margins while enforcing an across-session measure. This makes it a perfect fit for aforementioned biometric recognition problem. Comprehensive experiments on ECG and TEOAE variability due to time lapse and body posture are done. Performance of the proposed method is compared against seven state-of-the-art feature selection algorithms as well as another six approaches in the area of ECG and TEOAE biometric recognition. Experimental results demonstrate that the proposed method performs noticeably better than other algorithms.

Fingerprints have been extensively used for biometric recognition around the world. However, fingerprints are not secrets, and an adversary can synthesis a fake finger to spoof the biometric system. The mainstream of the current fingerprint spoof detection methods are basically binary classifier trained on some real and fake samples. While they perform well on detecting fake samples created by using the same methods used for training, their performance degrades when encountering fake samples created by a novel spoofing method. In this paper, we approach the problem from a different perspective by incorporating electrocardiogram (ECG). Compared with the conventional biometrics, stealing someone's ECG is far more difficult if not impossible. Considering that ECG is a vital signal and motivated by its inherent liveness, we propose to combine it with a fingerprint liveness detection algorithm. The combination is natural as both ECG and fingerprints can be captured from fingertips. In the proposed framework, the ECG and fingerprint are combined not only for authentication purpose but also for liveness detection. We also examine automatic template updating using ECG and fingerprint. In addition, we propose a stopping criterion that reduces the average waiting time for signal acquisition. We have performed extensive experiments on the LivDet2015 database which is presently the latest available liveness detection database and compare the proposed method with six liveness detection methods as well as 12 participants of LivDet2015 competition. The proposed system has achieved a liveness detection equal error rate (EER) of 4.2% incorporating only 5 s of ECG. By extending the recording time to 30 s, liveness detection EER reduces to 2.6% which is about 4 times better than the best of six comparison methods. This is also about 2 times better than the best results achieved by the participants of the LivDet2015 competition.

This paper proposes an anaesthesia monitoring system that accurately measures the depth of anaesthesia through 40-Hz auditory steady-state response. With accurate and fast depth of anaesthesia measuring, the monitor can reduce the incidence of awareness during surgical operation. The proposed denoising method for extracting 40-Hz auditory steady-state cycles, adaptive multilevel wavelet denoising, enabled the system to extract auditory steady-state response cycles from fewer epochs and over short periods of time which is of crucial importance in monitoring anaesthesia. The noise estimation scheme, adaptive threshold levels, rearranging, and multilevel denoising of frames increase the accuracy and signal to noise ratio of the extracted cycles. The modified fuzzy c-means clustering scheme, proposed to improve clustering performance in noisy data bases where no prior information about the level of noise and signal energy is available, is used for clustering the auditory steady-state cycles. Weighting the features with a novel algorithm and based on their differentiating role in clustering, the modified fuzzy c-means improves fuzziness in cluster partitions and the geometrical structure of the data. An index called depth of anaesthesia index is defined and determined at each cycle based on the clustering information of the cycle and the previous ones. The algorithm is applied to auditory steady-state response signals recorded from 20 human subjects during surgical operations with Propofol-induced general anaesthesia. The accuracy of the depth of anaesthesia index is validated through the subjects' medical markers, clinical parameters, and the recorded bispectral index during the induction phase. Depth of anaesthesia index is verified to be accurate and able to detect fast transitions between different levels of anaesthesia. The computed depth of anaesthesia indices detected the induction of anaesthesia on average 55 s faster than bispectral index and 17 s earlier than loss of eyelash reflex.

Conventional feature selection algorithms assign a single common feature set to all regions of the sample space. In contrast, this paper proposes a novel algorithm for localized feature selection for which each region of the sample space is characterized by its individual distinct feature subset that may vary in size and membership. This approach can therefore select an optimal feature subset that adapts to local variations of the sample space, and hence offer the potential for improved performance. Feature subsets are computed by choosing an optimal coordinate space so that, within a localized region, within-class distances and between-class distances are, respectively, minimized and maximized. Distances are measured using a logistic function metric within the corresponding region. This enables the optimization process to focus on a localized region within the sample space. A local classification approach is utilized for measuring the similarity of a new input data point to each class. The proposed logistic localized feature selection (lLFS) algorithm is invariant to the underlying probability distribution of the data; hence, it is appropriate when the data are distributed on a nonlinear or disjoint manifold. lLFS is efficiently formulated as a joint convex/increasing quasi-convex optimization problem with a unique global optimum point. The method is most applicable when the number of available training samples is small. The performance of the proposed localized method is successfully demonstrated on a large variety of data sets. We demonstrate that the number of features selected by the lLFS method saturates at the number of available discriminative features. In addition, we have shown that the Vapnik-Chervonenkis dimension of the localized classifier is finite. Both these factors suggest that the lLFS method is insensitive to the overfitting issue, relative to other methods.

Typical feature selection methods choose an optimal global feature subset that is applied over all regions of the sample space. In contrast, in this paper we propose a novel localized feature selection (LFS) approach whereby each region of the sample space is associated with its own distinct optimized feature set, which may vary both in membership and size across the sample space. This allows the feature set to optimally adapt to local variations in the sample space. An associated method for measuring the similarities of a query datum to each of the respective classes is also proposed. The proposed method makes no assumptions about the underlying structure of the samples; hence the method is insensitive to the distribution of the data over the sample space. The method is efficiently formulated as a linear programming optimization problem. Furthermore, we demonstrate the method is robust against the over-fitting problem. Experimental results on eleven synthetic and real-world data sets demonstrate the viability of the formulation and the effectiveness of the proposed algorithm. In addition we show several examples where localized feature selection produces better results than a global feature selection method.

The objective of a continuous authentication system is to continuously monitor the identity of subjects using biometric systems. In this paper, we proposed a novel feature extraction and a unique continuous authentication strategy and technique. We proposed One-Dimensional Multi-Resolution Local Binary Patterns (1DMRLBP), an online feature extraction for one-dimensional signals. We also proposed a continuous authentication system, which uses sequential sampling and 1DMRLBP feature extraction. This system adaptively updates decision thresholds and sample size during run-time. Unlike most other local binary patterns variants, 1DMRLBP accounts for observations' temporal changes and has a mechanism to extract one feature vector that represents multiple observations. 1DMRLBP also accounts for quantization error, tolerates noise, and extracts local and global signal morphology. This paper examined electrocardiogram signals. When 1DMRLBP was applied on the University of Toronto database (UofTDB) 1,012 single session subjects database, an equal error rate (EER) of 7.89% was achieved in comparison to 12.30% from a state-of-the-art work. Also, an EER of 10.10% was resulted when 1DMRLBP was applied to UofTDB 82 multiple sessions database. Experiments showed that using 1DMRLBP improved EER by 15% when compared with a biometric system based on raw time-samples. Finally, when 1DMRLBP was implemented with sequential sampling to achieve a continuous authentication system, 0.39% false rejection rate and 1.57% false acceptance rate were achieved.