Shubin Kim, Yejin Son, Junyeong Park, Keummin Ka, Seungbeen Lee, Jaeyoung Lee, Hyeju Jang, Alice Oh, Youngjae Yu. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Machine learning algorithms play a pivotal role in a wide range of Artificial Intelligence (AI) applications. Explaining the results and behavior of a machine learning model, however, remains a challenge. In this paper, we present a new approach to the explanation of machine learning models using a large language model (LLM). In this work, we seek natural language descriptions of the behavioral patterns of a machine learning model by a combination of prompting and model sampling. A subspace sampling technique is developed to generate ML model outputs using partial features in a user defined space. A projective visualization method is employed to guide the sampling process, including user-directed interactive sampling and feature-based sampling, so that an optimal amount of information can be provided to the LLM to ensure accurate and concise natural language explanations. Two public datasets, a student performance dataset and a weather dataset, were used to test our approach under various conditions.

To alleviate the teen vaping crisis, this study analyzed the "QuitVaping" subreddit. We identified posts sharing experience and/or advice using DistilBERT and investigated distinctive features in the data with topic modeling (BERTopic) and sentiment analysis (VADER) based on upvotes. We extracted 23 distinct vape-related symptoms and 17 suggested withdrawal methods, along with insightful topics. Highly upvoted posts often contained hopeful advice with positive sentiments. Our results help understand individual experiences, giving valuable insights to reduce teen vaping.

There has been increasing interest in non-invasive predictors of Alzheimer's disease (AD) as an initial screen for this condition. Previously, successful attempts leveraged eye-tracking and language data generated during picture narration and reading tasks. These results were obtained with high-end, expensive eye-trackers. Instead, we explore classification using eye-tracking data collected with a webcam, where our classifiers are built using a deep-learning approach. Our results show that the webcam gaze classifier is not as good as the classifier based on high-end eye-tracking data. However, the webcam-based classifier still beats the majority-class baseline classifier in terms of AU-ROC, indicating that predictive signals can be extracted from webcam gaze tracking. Hence, although our results indicate that there is still a long way to go before webcam gaze tracking can reach practical relevance, they still provide an encouraging proof of concept that this technology should be further explored as an affordable alternative to high-end eye-trackers for the detection of AD.

Much research has been done within the social sciences on the interpretation and influence of stigma on human behaviour and health, which result in out-of-group exclusion, distancing, cognitive separation, status loss, discrimination, in-group pressure, and often lead to disengagement, non-adherence to treatment plan, and prescriptions by the doctor. However, little work has been conducted on computational identification of stigma in general and in social media discourse in particular. In this paper, we develop the annotation scheme and improve the annotation process for stigma identification, which can be applied to other health-care domains. The data from pro-vaccination and anti-vaccination discussion groups are annotated by trained annotators who have professional background in social science and health-care studies, therefore the group can be considered experts on the subject in comparison to non-expert crowd. Amazon MTurk annotators is another group of annotator with no knowledge on their education background, they are initially treated as non-expert crowd on the subject matter of stigma. We analyze the annotations with visualisation techniques, features from LIWC (Linguistic Inquiry and Word Count) list and make prediction based on bi-grams with traditional and deep learning models. Data augmentation method and application of CNN show high performance accuracy in comparison to other models. Success of the rigorous annotation process on identifying stigma is reconfirmed by achieving high prediction rate with CNN.

Much research has been done within the social sciences on the interpretation and influence of stigma on human behaviour and health, which result in out-of-group exclusion, distancing, cognitive separation, status loss, discrimination, in-group pressure, and often lead to disengagement, non-adherence to treatment plan, and prescriptions by the doctor. However, little work has been conducted on computational identification of stigma in general and in social media discourse in particular. In this paper, we develop the annotation scheme for stigma based on social science theories, and perform a corpus study on the data from Facebook groups on vaccination. The data from pro-vaccination and anti-vaccination discussion groups are annotated by trained annotators and by MTurk annotators. We analyze the annotations using LIWC (Linguistic Inquiry and Word Count) software and TF-IDF in order to identify differentiating features between stigmatizing vs. non-stigmatizing content. Our corpus study lays a valuable foundation in computational modeling of social stigma, as it can serve as validation/interpretation of the social science theories through the prism of laypeople understanding. The annotated corpora can be subsequently used for automatic stigma identification. Moreover, the annotation scheme can be applied to study stigma across different themes and diseases and can be utilized during public health informational campaigns and health interventions.

Facebook often serves as a platform for sharing health-related information and is a venue to express attitudes, thoughts, and frustrations within groups centered around healthcare themes. This information can be utilized for public health monitoring, with the aim of tackling stigmatized and stereotypical attitudes in relation to immunization or other health related issues expressed in social media. However, the effectiveness of those attempts will rest on our understanding of the concept of stigma and its correct modeling. In this study, we aim to expand the small pool of existing computational studies on the topic of stigma identification in a health care context. More specifically, we compare the following models using a dataset of 2,761 comments from Facebook: Convolutional Neural Network (CNN): Term Frequency-Inverse Document Frequency (TF-IDF) with Logistic Regression (LR), Support Vector Machine (SVM), Naive Bayes (NB), Multilayer Perceptron (MLP), Random Forest (RF), K-nearest neighbours (KNN), and Stochastic Gradient Descent (SGDC), Long short-term memory networks (LSTM), Bidirectional long short-term memory (BiLSTM), and fastText. Accuracy results as evaluated on an unbalanced data subset (with limited training samples) show that fastText gives the best performance, although BiLSTM and CNN achieve comparably good results on unbalanced data as well. CNN algorithm significantly outperforms other algorithms on balanced version of the dataset according to a paired sample t-test ( ).

We present an unsupervised model of dialogue act sequences in conversation. By modeling topical themes as transitioning more slowly than dialogue acts in conversation, our model de-emphasizes content-related words in order to focus on conversational function words that signal dialogue acts. We also incorporate speaker tendencies to use some acts more than others as an additional predictor of dialogue act prevalence beyond temporal dependencies. According to the evaluation presented on two dissimilar corpora, the CNET forum and NPS Chat corpus, the effectiveness of each modeling assumption is found to vary depending on characteristics of the data. De-emphasizing content-related words yields improvement on the CNET corpus, while utilizing speaker tendencies is advantageous on the NPS corpus. The components of our model complement one another to achieve robust performance on both corpora and outperform state-of-the-art baseline models.

In this paper, we present a novel and highly effective method for induction and application of metaphor frame templates as a step toward detecting metaphor in extended discourse.We infer implicit facets of a given metaphor frame using a semisupervised bootstrapping approach on an unlabeled corpus.Our model applies this frame facet information to metaphor detection, and achieves the state-of-the-art performance on a social media dataset when building upon other proven features in a nonlinear machine learning model.In addition, we illustrate the mechanism through which the frame and topic information enable the more accurate metaphor detection.

Abstract We describe the development, pilot-testing, refinement, and four evaluations of Diagnostic Question Generator (DQGen), which automatically generates multiple choice cloze (fill-in-the-blank) questions to test children's comprehension while reading a given text. Unlike previous methods, DQGen tests comprehension not only of an individual sentence but of the context preceding it. To test different aspects of comprehension, DQGen generates three types of distractors: ungrammatical distractors test syntax; nonsensical distractors test semantics; and locally plausible distractors test inter-sentential processing. (1) A pilot study of DQGen 2012 evaluated its overall questions and individual distractors, guiding its refinement into DQGen 2014. (2) Twenty-four elementary students generated 200 responses to multiple choice cloze questions that DQGen 2014 generated from forty-eight stories. In 130 of the responses, the child chose the correct answer. We define thedistractivenessof a distractor as the frequency with which students choose it over the correct answer. The incorrect responses were consistent with expected distractiveness: twenty-seven were plausible, twenty-two were nonsensical, fourteen were ungrammatical, and seven were null. (3) To compare DQGen 2014 against DQGen 2012, five human judges categorized candidate choices without knowing their intended type or whether they were the correct answer or a distractor generated by DQGen 2012 or DQGen 2014. The percentage of distractors categorized as their intended type was significantly higher for DQGen 2014. (4) We evaluated DQGen 2014 against human performance based on 1,486 similarly blind categorizations by twenty-seven judges of sixteen correct answers, forty-eight distractors generated by DQGen 2014, and 504 distractors authored by twenty-one humans. Surprisingly, DQGen 2014 did significantly better than humans at generating ungrammatical distractors and marginally better than humans at generating nonsensical distractors, albeit slightly worse at generating plausible distractors. Moreover, vetting DQGen 2014's output and writing distractors only when necessary would halve the time to write them all, and produce higher quality distractors.

Metaphor is a common linguistic tool in communication, making its detection in discourse a crucial task for natural language understanding.One popular approach to this challenge is to capture semantic incohesion between a metaphor and the dominant topic of the surrounding text.While these methods are effective, they tend to overclassify target words as metaphorical when they deviate in meaning from its context.We present a new approach that (1) distinguishes literal and non-literal use of target words by examining sentence-level topic transitions and (2) captures the motivation of speakers to express emotions and abstract concepts metaphorically.Experiments on an online breast cancer discussion forum dataset demonstrate a significant improvement in metaphor detection over the state-of-theart.These experimental results also reveal a tendency toward metaphor usage in personal topics and certain emotional contexts.

Understanding contextual information is key to detecting metaphors in discourse.Most current work aims at detecting metaphors given a single sentence, thus focusing mostly on local contextual cues within a short text.In this paper, we present a novel approach that explicitly leverages global context of a discourse to detect metaphors.In addition, we show that syntactic information such as dependency structures can help better describe local contextual information, thus improving detection results when combined.We apply our methods on a newly annotated online discussion forum, and show that our approach outperforms the state-of-the-art baselines in previous literature.