Today digital revolution is having a dramatic impact on the pharmaceutical industry and the entire healthcare system. The implementation of machine learning, extreme-scale computer simulations, and big data analytics in the drug design and development process offers an excellent opportunity to lower the risk of investment and reduce the time to the patient.

Game-based learning environments have the distinctive capacity to promote learning experiences that are both engaging and effective. Recent advances in sensor-based technologies (e.g., facial expression analysis and eye gaze tracking) and natural language processing have introduced the opportunity to leverage multimodal data streams for learning analytics. Learning analytics and student modeling informed by multimodal data captured during students’ interactions with game-based learning environments hold significant promise for designing effective learning environments that detect unproductive student behaviors and provide adaptive support for students during learning. Learning analytics frameworks that can accurately predict student learning outcomes early in students’ interactions hold considerable promise for enabling environments to dynamically adapt to individual student needs. In this paper, we investigate a multimodal, multi-task predictive student modeling framework for game-based learning environments. The framework is evaluated on two datasets of game-based learning interactions from two student populations (n=61 and n=118) who interacted with two versions of a game-based learning environment for microbiology education. The framework leverages available multimodal data channels from the datasets to simultaneously predict student post-test performance and interest. In addition to inducing models for each dataset individually, this work investigates the ability to use information learned from one source dataset to improve models based on another target dataset (i.e., transfer learning using pre-trained models). Results from a series of ablation experiments indicate the differences in predictive capacity among a combination of modalities including gameplay, eye gaze, facial expressions, and reflection text for predicting the two target variables. In addition, multi-task models were able to improve predictive performance compared to single-task baselines for one target variable, but not both. Lastly, transfer learning showed promise in improving predictive capacity in both datasets.

Inability to regulate affective states can impact one's capacity to engage in higher-order thinking like scientific reasoning with game-based learning environments. Many efforts have been made to build affect-aware systems to mitigate the potentially detrimental effects of negative affect. Yet, gaps in research exist since accurately capturing and modeling affect as a state that changes dynamically over time is methodologically and analytically challenging. In this paper, we calculated multilevel mixed effects growth models to assess whether seventy-eight participants’ (n= 78) time engaging in scientific reasoning (via logfiles and eye gaze) were related to time facially expressing confused, frustrated, and neutral states (via facial recognition software) during game-based learning with Crystal Island. The fitted model estimated significant positive relations between the time learners facially expressed confusion, frustration, and neutral states and time engaging in scientific-reasoning actions. The time individual learners facially expressed frustrated, confused, and neutral states explained a significant amount of variation in time engaging in scientific reasoning. Our finding emphasize that individual differences and agency may play a important role on relations between affective states, their dynamics, and higher-order cognition during game-based learning. Designing affect-aware game-based learning environments that track the dynamics within individual learners’ affective states may best support cognition.

Block-based programming environments are widely used by novices who are learning computer science. However, even in block-based coding environments that have been carefully developed to serve novices, students frequently struggle and require additional support. A promising avenue to provide this support is the use of intelligent tutoring systems, which offer adaptive hints to assist learners. In order to provide students with the adaptive hints they need, we must investigate their help-seeking behaviors and identify patterns surrounding their need for support. In this experience report, we examine data collected from 174 college students in an introductory engineering course, who used an intelligent block-based coding environment to learn computer science. These students made more than 1,000 hint requests, which we represent in two-dimensional space along axes of elapsed time and code completeness. Analysis revealed five major clusters of hint requests, which we further characterized through qualitative examination of the coding trajectories that preceded each hint request. We also analyzed how students' incoming knowledge and perceived computer skill were related to their help-seeking behaviors. Students with higher incoming knowledge requested hints when their code was more complete than students with lower incoming knowledge. Students with high perceived computer skill asked for hints when their code was less complete than those with low perceived computer skill. The results presented here provide insight into student help-seeking behavior in computer science education, informing CS educators and system designers on how best to develop support strategies.

Block-based programming environments are widely used in computer science education. However, these environments pose significant challenges for student modeling. Given a series of problem-solving actions taken by students in block-based programming environments, student models need to accurately infer problem-solving students’ programming abilities in real time to enable adaptive feedback and hints that are tailored to students’ abilities. While student models for block-based programming offer the potential to support student-adaptivity, creating student models for these environments is challenging because students can develop a broad range of solutions to a given programming activity. To address these challenges, we introduce a progression trajectory-based student modeling framework for modeling novice student block-based programming across multiple learning activities. Student trajectories utilize a time series representation that employs code analysis to incrementally compare student programs to expert solutions as students undertake block-based programming activities. This paper reports on a study in which progression trajectories were collected from more than 100 undergraduate students engaging in a series of block-based programming activities in an introductory computer science course. Using progression trajectory-based student modeling, we identified three distinct trajectory classes: Early Quitting, High Persistence, and Efficient Completion. Analysis of these trajectories revealed that they exhibit significantly different characteristics with respect to students’ actions and can be used to accurately predict students’ programming behaviors on future programming activities compared to competing baseline models. The findings suggest that progression trajectory-based student models can accurately model students’ block-based programming problem solving and hold potential for informing adaptive support in block-based programming environments.

Modeling student knowledge is critical in adaptive learning environments. Predictive student modeling enables formative assessment of student knowledge and skills, and it drives personalized support to create learning experiences that are both effective and engaging. Traditional approaches to predictive student modeling utilize features extracted from students’ interaction trace data to predict student test performance, aggregating student test performance as a single output label. We reformulate predictive student modeling as a multi-task learning problem, modeling questions from student test data as distinct “tasks.” We demonstrate the effectiveness of this approach by utilizing student data from a series of laboratory-based and classroom-based studies conducted with a game-based learning environment for microbiology education, Crystal Island. Using sequential representations of student gameplay, results show that multi-task stacked LSTMs with residual connections significantly outperform baseline models that do not use the multi-task formulation. Additionally, the accuracy of predictive student models is improved as the number of tasks increases. These findings have significant implications for the design and development of predictive student models in adaptive learning environments.

Modeling visitor engagement is a key challenge in informal learning environments, such as museums and science centers. Devising predictive models of visitor engagement that accurately forecast salient features of visitor behavior, such as dwell time, holds significant potential for enabling adaptive learning environments and visitor analytics for museums and science centers. In this paper, we introduce a multimodal early prediction approach to modeling visitor engagement with interactive science museum exhibits. We utilize multimodal sensor data including eye gaze, facial expression, posture, and interaction log data captured during visitor interactions with an interactive museum exhibit for environmental science education, to induce predictive models of visitor dwell time. We investigate machine learning techniques (random forest, support vector machine, Lasso regression, gradient boosting trees, and multi-layer perceptron) to induce multimodal predictive models of visitor engagement with data from 85 museum visitors. Results from a series of ablation experiments suggest that incorporating additional modalities into predictive models of visitor engagement improves model accuracy. In addition, the models show improved predictive performance over time, demonstrating that increasingly accurate predictions of visitor dwell time can be achieved as more evidence becomes available from visitor interactions with interactive science museum exhibits. These findings highlight the efficacy of multimodal data for modeling museum exhibit visitor engagement.

Recent years have seen an increasing interest in identifying common student misconceptions during introductory programming. In a parallel development, block-based programming environments for novice programmers have grown in popularity, especially in introductory courses. While these environments eliminate many syntax-related errors faced by novice programmers, there has been limited work that investigates the types of misconceptions students might exhibit in these environments. Developing a better understanding of these misconceptions will enable these programming environments and instructors to more effectively tailor feedback to students, such as prompts and hints, when they face challenges. In this paper, we present results from a cluster analysis of student programs from interactions with programming activities in a block-based programming environment for introductory computer science education. Using the interaction data from students' programming activities, we identify three families of student misconceptions and discuss their implications for refinement of the activities as well as design of future activities. We then examine the value of block counts, block sequence counts, and system interaction counts as programming features for clustering block-based programs. These clusters can help researchers identify which students would benefit from feedback or interventions and what kind of feedback provides the most benefit to that particular student.

Recent years have seen a growing interest in block-based programming environments for computer science education. Although block-based programming offers a gentle introduction to coding for novice programmers, introductory computer science still presents significant challenges, so there is a great need for block-based programming environments to provide students with adaptive support. Predictive student modeling holds significant potential for adaptive support in block-based programming environments because it can identify early on when a student is struggling. However, predictive student models often make a number of simplifying assumptions, such as assuming a normal response distribution or homogeneous student characteristics, which can limit the predictive performance of models. These assumptions, when invalid, can significantly reduce the predictive accuracy of student models.

Utilizing sensors for affect detection in adaptive learning technologies has been the subject of growing interest in recent years. This extends to the collection of multiple concurrent sensor-based input channels to enable multimodal affective modeling. However, sensors pose significant challenges to affect detection, including sensor connectivity issues, background noise, inconsistent data logging, and loss of data due to hardware failure. In this paper, we introduce a framework for multimodal data imputation to improve automated detection of student affect in adaptive learning technologies. Through the use of an autoencoder neural network trained on Microsoft Kinect-based posture data and electrodermal activity data with synthetic noise injection, we approximate missing values within the original dataset while still preserving the inter-related context between features when reconstructing the dataset. The reconstructed dataset can be used in conjunction with multimodal data fusion techniques to further boost affect detector accuracy. Results indicate that this framework improves the effectiveness of multimodal affect detectors when compared to unimodal baseline models, as well as models using baseline data imputation techniques such as mean imputation. Further, it maintains cross-modality information that influences the multimodal affect detectors' performance, as the approach also outperforms previous work using the latent representation of the imputed dataset as training data instead of a complete reconstruction of the original dataset's dimensionality.

Recent advances in eye-tracking technologies have introduced the opportunity to incorporate gaze into student modeling. Creating student models that leverage gaze information holds significant promise for game-based learning environments. This paper introduces a gaze-enhanced student modeling framework that incorporates student eye tracking to dynamically predict students' performance in a game-based learning environment for microbiology education, CRYSTAL ISLAND. The gaze-enhanced student modeling framework was investigated in a study comparing a gaze-enhanced student model with a baseline student model that does not utilize student eye-tracking. Results of a study conducted with 65 college students interacting with the CRYSTAL ISLAND game-based learning environment indicate that the gaze-enhanced student model significantly outperforms the baseline model in dynamically predicting student problem-solving performance. The findings suggest that incorporating gaze into student modeling can contribute to a new generation of student models for game-based learning environments.