This work-in-progress study explores the use of Large Language Models (LLMs) to dynamically personalize datasets used in data science education according to learner interests. The study outlines two dataset personalization methods that transform each variable in the original dataset to a new variable: a scaling method that transforms while preserving the original distributions and correlation structure, and a sampling method that transforms while preserving the original correlation structure but changes distributions to match the new variables. An evaluation with subject matter experts revealed that datasets personalized using the scaling method were not significantly different from the original datasets in terms of the appropriateness of variable names and ranges. Further evaluation using the personalized datasets in the context of instructional materials designed for the original datasets indicated that more inconsistencies were found with the sampling method than the scaling method. These results suggest that dataset personalization can create datasets that serve as drop-in replacements for the original datasets in existing instructional materials.

The workshop will give educators an introduction to graphical data analysis techniques for exploring, summarizing, and effectively communicating data using Visual Blocks (Blockly) and Jupiter Notebooks. Participants will gain skills to create and interpret various graphs and plots, fostering insights into data patterns and relationships. The emphasis will be on adeptly selecting visualizations. Upon completion of this course, educators should be proficient in: Understanding the principles of graphical data analysis and their practical applications; and creating and interpreting diverse graph types and plots. The workshop will further delve into statistical methods essential for data analysis, guiding educators on employing descriptive statistics to explore data and derive meaningful business insights. The focus will be on fostering an understanding of statistical concepts and their practical application, as well as the effective interpretation and communication of findings. Additionally, the program will introduce educators to machine learning regressors, with a primary focus on linear regression. Participants will learn the skills to train, evaluate, and apply regressors to predict continuous target variables from input features. The emphasis will be on grasping and applying machine learning principles, as well as effectively interpreting and communicating model predictions. Furthermore, participants will engage in hands-on experience with machine learning classifiers, encompassing logistic regression, decision trees, naive Bayes, and neural networks. Educators will acquire the expertise to train, evaluate, and apply classifiers for predicting categorical target variables from input features. The emphasis here is on understanding and applying machine learning principles, as well as adeptly interpreting model predictions.

In contrast to simple feedback, which provides students with the correct answer, elaborated feedback provides an explanation of the correct answer with respect to the student's error. Elaborated feedback is thus a challenge for AI in education systems because it requires dynamic explanations, which traditionally require logical reasoning and knowledge engineering to generate. This study presents an alternative approach that formulates elaborated feedback in terms of long-form question answering (LFQA). An off-the-shelf LFQA system was evaluated by human raters in a 2x2x2x2 ablation design that manipulated the context documents given to the LFQA model and the post-processing of model output. Results indicate that context manipulations improve performance but that post-processing can have detrimental results.

Blocks languages are widely used to teach children programming, and research over the past decade has generally supported their benefits in terms of motivation and actual learning. However, little work has been done on using blocks languages to teach adults, and even less work has looked at blocks languages for data science. We have integrated Blockly, a blocks-based programming environment, into JupyterLab, one of the leading computational notebook development environments for data science. Our integration, which is publicly released as a JupyterLab extension, allows users to assemble blocks-based programs in a GUI workspace and then render the blocks as textual code (e.g., Python) in a computational notebook cell. Additional features of the extension are notebook sync, which clears and restores the blocks workspace as the user navigates to different notebook cells, and intelliblocks, dynamically generated blocks that are created when users load software packages. In this demonstration, we show how learners can use blocks to solve data science problems in a JupyterLab computational notebook. We have released 20 worked example companion notebooks at https://github.com/memphis-iis/datawhys-content-notebooks and an extended tutorial on the extension at https://youtu.be/-luPzplPDI0. This material is based upon work supported by the National Science Foundation under Grant No. 1918751.

We investigate automatic detection of teacher questions from audio recordings collected in live classrooms with the goal of providing automated feedback to teachers. Using a dataset of audio recordings from 11 teachers across 37 class sessions, we automatically segment the audio into individual teacher utterances and code each as containing a question or not. We train supervised machine learning models to detect the human-coded questions using high-level linguistic features extracted from automatic speech recognition (ASR) transcripts, acoustic and prosodic features from the audio recordings, as well as context features, such as timing and turn-taking dynamics. Models are trained and validated independently of the teacher to ensure generalization to new teachers. We are able to distinguish questions and non-questions with a weighted F1 score of 0.69. A comparison of the three feature sets indicates that a model using linguistic features outperforms those using acoustic-prosodic and context features for question detection, but the combination of features yields a 5% improvement in overall accuracy compared to linguistic features alone. We discuss applications for pedagogical research, teacher formative assessment, and teacher professional development.

Current learning technologies have no direct way to assess students' mental effort: are they in deep thought, struggling to overcome an impasse, or are they zoned out? To address this challenge, we propose the use of EEG-based cognitive load detectors during learning. Despite its potential, EEG has not yet been utilized as a way to optimize instructional strategies. We take an initial step towards this goal by assessing how experimentally manipulated (easy and difficult) sections of an intelligent tutoring system (ITS) influenced EEG-based estimates of students' cognitive load. We found a main effect of task difficulty on EEG-based cognitive load estimates, which were also correlated with learning performance. Our results show that EEG can be a viable source of data to model learners' mental states across a 90-minute session.

We investigate multi-sensor modeling of teachers’ instructional segments (e.g., lecture, group work) from audio recordings collected in 56 classes from eight teachers across five middle schools. Our approach fuses two sensors: a unidirectional microphone for teacher audio and a pressure zone microphone for general classroom audio. We segment and analyze the audio streams with respect to discourse timing, linguistic, and paralinguistic features. We train supervised classifiers to identify the five instructional segments that collectively comprised a majority of the data, achieving teacher-independent F1 scores ranging from 0.49 to 0.60. With respect to individual segments, the individual sensor models and the fused model were on par for Question & Answer and Procedures & Directions segments. For Supervised Seatwork, Small Group Work, and Lecture segments, the classroom model outperformed both the teacher and fusion models. Across all segments, a multi-sensor approach led to an average 8% improvement over the state of the art approach that only analyzed teacher audio. We discuss implications of our findings for the emerging field of multimodal learning analytics.

Nathaniel Blanchard, Patrick Donnelly, Andrew M. Olney, Borhan Samei, Brooke Ward, Xiaoyi Sun, Sean Kelly, Martin Nystrand, Sidney K. D’Mello. Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue. 2016.

We investigate automatic analysis of teachers' instructional strategies from audio recordings collected in live classrooms. We collected a data set of teacher audio and human-coded instructional activities (e.g., lecture, question and answer, group work) in 76 middle school literature, language arts, and civics classes from eleven teachers across six schools. We automatically segment teacher audio to analyze speech vs. rest patterns, generate automatic transcripts of the teachers' speech to extract natural language features, and compute low-level acoustic features. We train supervised machine learning models to identify occurrences of five key instructional segments (Question & Answer, Procedures and Directions, Supervised Seatwork, Small Group Work, and Lecture) that collectively comprise 76% of the data. Models are validated independently of teacher in order to increase generalizability to new teachers from the same sample. We were able to identify the five instructional segments above chance levels with F1 scores ranging from 0.64 to 0.78. We discuss key findings in the context of teacher modeling for formative assessment and professional development.

We focus on data collection designs for the automated analysis of teacher-student interactions in live classrooms with the goal of identifying instructional activities (e.g., lecturing, discussion) and assessing the quality of dialogic instruction (e.g., analysis of questions). Our designs were motivated by multiple technical requirements and constraints. Most importantly, teachers could be individually micfied but their audio needed to be of excellent quality for automatic speech recognition (ASR) and spoken utterance segmentation. Individual students could not be micfied but classroom audio quality only needed to be sufficient to detect student spoken utterances. Visual information could only be recorded if students could not be identified. Design 1 used an omnidirectional laptop microphone to record both teacher and classroom audio and was quickly deemed unsuitable. In Designs 2 and 3, teachers wore a wireless Samson AirLine 77 vocal headset system, which is a unidirectional microphone with a cardioid pickup pattern. In Design 2, classroom audio was recorded with dual first- generation Microsoft Kinects placed at the front corners of the class. Design 3 used a Crown PZM-30D pressure zone microphone mounted on the blackboard to record classroom audio. Designs 2 and 3 were tested by recording audio in 38 live middle school classrooms from six U.S. schools while trained human coders simultaneously performed live coding of classroom discourse. Qualitative and quantitative analyses revealed that Design 3 was suitable for three of our core tasks: (1) ASR on teacher speech (word recognition rate of 66% and word overlap rate of 69% using Google Speech ASR engine); (2) teacher utterance segmentation (F-measure of 97%); and (3) student utterance segmentation (F-measure of 66%). Ideas to incorporate video and skeletal tracking with dual second-generation Kinects to produce Design 4 are discussed.