A key to personalized online learning is presenting content at an appropriate difficulty level; content that is too difficult can cause frustration and content that is too easy may result in boredom. Appropriate content can improve students' engagement and learning outcome. In this research, we propose a computer vision enhanced problem selector (COVES), a deep learning model to select a personalized difficulty level for each student. A combination of visual information and traditional log data is used to predict student-problem interactions, which are then used to guide problem difficulty selection in real time. COVES was trained on a dataset of fifty-one sixth-grade students interacting with the online math tutor MathSpring. Once COVES was integrated into the tutor, its effectiveness was tested with twenty-two seventh-grade students in controlled experiments. Students who received problems at an appropriate difficulty level, based on real-time predictions of their performance, demonstrated improved engagement with the math tutor. Results indicate that COVES leads to higher mastery of math concepts, better timing, and higher scores, thus providing a positive learning experience for the participants.

In this work, we propose a video-based transfer learning approach for predicting problem outcomes of students working with an intelligent tutoring system (ITS). By analyzing a student's face and gestures, our method predicts the outcome of a student answering a problem in an ITS from a video feed. Our work is motivated by the reasoning that the ability to predict such outcomes enables tutoring systems to adjust interventions, such as hints and encouragement, and to ultimately yield improved student learning. We collected a large labeled dataset of student interactions with an intelligent online math tutor consisting of 68 sessions, where 54 individual students solved 2,749 problems. We will release this dataset publicly upon publication of this paper. It will be available at https://www.cs.bu.edu/faculty/betke/research/learning/. Working with this dataset, our transfer-learning challenge was to design a representation in the source domain of pictures obtained “in the wild” for the task of facial expression analysis, and transferring this learned representation to the task of human behavior prediction in the domain of webcam videos of students in a classroom environment. We developed a novel facial affect representation and a user-personalized training scheme that unlocks the potential of this representation. We designed several variants of a recurrent neural network that models the temporal structure of video sequences of students solving math problems. Our final model, named ATL-BP for Affect Transfer Learning for Behavior Prediction, achieves a relative increase in mean F -score of 50 % over the state-of-the-art method on this new dataset.

The development of new technologies at an unprecedented rate is rapidly changing the landscape of the labor market. Therefore, for workers who want to build a successful career, acquiring new skills required by new jobs through lifelong learning is crucial. In this paper, we propose a novel and interpretable monotonic nonlinear state-space model to analyze online user professional profiles and provide actionable feedback and recommendations to users on how they can reach their career goals. Specifically, we use a series of binary-valued and non-decreasing latent states to represent the expanding skill set of each user throughout their career and propose an efficient inference method under our model. Using a series of experiments on two large real-world datasets, we show that our model (sometimes significantly) outperforms existing methods on the tasks of company, job title, and skill prediction. More importantly, our model is interpretable and can be used for other important tasks including skill gap identification and career path planning. Using a series of case studies, we show that our model can provide i) actionable feedback to users and guide them through their upskilling and reskilling processes and ii) recommendations of feasible paths for users to reach their career goals.

Equipping an Intelligent Tutoring System (ITS) with the ability to interpret affective signals from students could potentially improve the learning experience of students by enabling the tutor to monitor the students' progress and provide timely interventions as well as present appropriate affective reactions via a virtual tutor. Most ITSs equipped with affect modeling capabilities attempt to predict the emotional state of users. However, the focus in this work is instead on trying to directly predict the learning outcomes of students from a stream of video capturing the students faces as they work on a set of math problems. Using facial features extracted from a video stream, we train classifiers to directly predict the success or failure of a student's attempt to answer a question while the student has just begun to work on the problem. In this work, we first introduce a novel dataset of student interactions with MathSpring, a popular ITS. We provide an exploratory analysis of the different problem outcome classes using typical facial action unit activations. We develop baseline models to predict the problem outcome labels of students solving math problems and discuss how early problem outcome labels can be forecasted and utilized to provide possible interventions.

This paper describes a new method for helping students improve their ability to develop proofs, a skill necessary for comprehending and appreciating the foundational topics of computer science. Our method transforms ordinary pen-and-paper homework problems into a puzzle-like game, where students connect dots to justify assertions, in a quest to reach a desired goal. We have implemented a software tutoring system using this method, for students to use at home as an optional study aid. Potentially, our system could one day become a full replacement for traditional hand-written homework, which has the additional benefit for course instructors of automating the grading of student work. Our system is also easy to adapt to any class that requires students to write proofs, and it is easy for instructors to create new problems to use with this system. This stands in contrast to many other educational tools for teaching proofs, which are limited to specific topic domains. We have demonstrated the versatility of our system by testing it in two computer science classes at a large public university. One was a Sophomore-level discrete mathematics course where the students were learning first-order prepositional logic, and the other was a Junior-level algorithms course where students were being first exposed to the concept of NP-completeness. Students from our experiments reported that they would like our system to be used in more of their classes.

Students self-reported not only their emotional state, but also the causal attributions of their emotions. After coding emotions with internal references to self, and external references to the environment or domain, we examined how sub-groups of students based on internal/external attributions and above or below median performance differ in terms of their emotional state, perceptions of item difficulty, and gender.

We report on the reactions of males and female students to the presence of animated pedagogical agents that provided emotional and motivational support. One hundred high school students used agents embedded in an Intelligent Tutoring System for Mathematics and randomized controlled evaluations compared students with and without learning companions. The results indicate that affective pedagogical agents improve affective outcomes of students in general and particularly so for female students, who reported being more frustrated and less confident while solving math problems prior to using the tutoring system. We discuss issues of incorporating gender into user models and of generating responses tailored to gender.

Many children's games have a universal, cross-cultural appeal and have been played for hundreds of years, suggesting a developmental need for them. This paper presents a framework for leveraging this appeal and longevity by developing cognitive games based on popular playground games. This framework employs technology to maintain the physicality and embodied components of the game. The Prête-à-apprendre (PAP+) toolkit is used to develop an e-Textiles game for teaching concepts of estimations and number sense in Mathematics. The game consists of "hoodies" and "barrels" that communicate with each other using a low-power wireless network (Zigbee). Each "hoodie" contains an LED display to convey constraints to children while each barrel contains and Radio Frequency Identifier (RFID) reader to identify children. The game can be deployed in structured or unstructured environments. An evaluation of the pedagogical design of the game based on various intrinsic and extrinsic motivation criteria is also presented.

We describe how web service architectures can provide better performance to applications by offering fine-grained services. We define web service granularity in terms of the amount of data that can be retrieved from a service in a single request on average. This is important because developers cannot predict if students will be using state of the art hardware. Thus, service-oriented architectures (SOA) with fine service granularity can minimize network communication and allow server machines to perform more work for applications. We present the Rashi Intelligent Tutoring System and describe how its architecture has been adapted into a web service with two competing application interfaces. We show how the interface that uses more fine-grained services leads to significant improvements in network message response time, message size, and response size, without a significant change in the number of requests.

We present an educational 3D game called Rashi Game, which instructs students via exploration using the inquiry teaching method. We first describe the Rashi Intelligent Tutoring System in its original form, and then describe the details and features of Rashi Game, the 3D game that we have developed. In particular, we argue that inquiry-learning environments are particularly viable for educational 3D games. This is because of the similarities between some common game mechanics and the inquiry-learning paradigm. These include freedom to explore open-ended environments, interaction with environments, and realistic scenarios. We briefly summarizing results that have been obtained via pilot studies of the efficacy of Rashi Game, and remark on what directions future work in educational 3D games might take.

We describe an educational 3D game called Rashi Game. Rashi Game features a fully functional, and open-ended, 3D environment for students backed by a domain-independent inquiry-learning tutor. We present pilot work that directly compares Rashi's classic 2D interface against the 3D game environment. We compare both the student's work within the system, as well as their reported sense of presence. Specifically, we notice some interesting patterns in student behavior within the game, dependent on the student's preference for games, and argue that there may be potential for modeling when and how to present a student with an educational game based on simple factors such as whether or not the student plays games regularly, or based on student affect (e.g. a lack of motivation).

This qualitative study presented high school students' help-seeking behavior and how they interacted with hints while they solved math problems on an intelligent tutoring system for math.

This paper describes the use of sensors in intelligent tutors to detect students' affective states and to embed emotional support. Using four sensors in two classroom experiments the tutor dynamically collected data streams of physiological activity and students' self-reports of emotions. Evidence indicates that state-based fluctuating student emotions are related to larger, longer-term affective variables such as self-concept in mathematics. Students produced self-reports of emotions and models were created to automatically infer these emotions from physiological data from the sensors. Summaries of student physiological activity, in particular data streams from facial detection software, helped to predict more than 60% of the variance of students emotional states, which is much better than predicting emotions from other contextual variables from the tutor, when these sensors are absent. This research also provides evidence that by modifying the “context” of the tutoring system we may well be able to optimize students' emotion reports and in turn improve math attitudes.

We researched the impact of gendered pedagogical agents on student attitudes for math, motivation and achievement in math, within the context of an adaptive tutoring software for high school mathematics. Learning companions emphasize perseverance by valuing effort in challenging tasks. They are also empathetic, as they reflect students' emotional states. The results suggest that, across two studies, it was the male learning companion that produced the most positive impact on female students' state-based emotions, attitudes and learning. It is possible that girls transfer their stereotypes to the computer software.

This research seeks to improve learning by integrating intelligent coaching with peer-to-peer collaboration. We combine advanced technologies that support student freedom and exploration, allow for collaboration and peer tutoring, and provide task advice by modeling student work and behavior. We empirically evaluated both the effect of added collaborative features that enable peer-to-peer interaction and the potential to use this collaboration to improve the intelligent coaching features of the system. We found that collaboration enhances student work and that the coaching features provide opportunities to promote collaboration.

We describe a novel framework developed for transfer learning within reinforcement learning (RL) problems. Then we exhibit how this framework can be extended to intelligent tutoring systems (ITS). We compose an algorithm that automatically constructs a graphical representation based on the transfer framework. We evaluate this on a real-world ITS example and show that the model constructed by our approach performs better than previously published results. We propose that transfer learning is a useful and related area to explore for furthering intelligent tutoring systems.

Knowledge representation remains a serious issue for researchers of intelligent tutoring systems. Two areas of knowledge representation that are particularly difficult are domain and teaching knowledge. This article discusses and gives example solutions to these knowledge engineering issues and also addresses issues that relate to up‐scaling existing intelligent tutoring technology to practical levels so that tutoring systems can be brought into the real world.

The introductory programming course at this university attempts to serve some 1500 students each semester. The attrition rate, due in part to the overload on the system and in part to the students' difficulties in “keeping up”, has, at times, approached 25%. In response to this situation we have revised and reordered the curriculum for use in an experimental course designed for the novice user. The course is directed toward discovering and addressing the confusions of new programming students. It facilitates our ongoing study of the novice programmers' response to graphics, friendly interface packages and the revised curriculum which includes the teaching of procedures and control structures at the beginning of the course. In studying these responses we have learned some techniques in aiding the novice user to unravel some of the mysteries surrounding the acquisition of programming skills. The course is constantly undergoing development in addition to being in its second semester as a departmental offering. It is detailed in this paper.

An effective human-computer discourse system requires more than a clever grammar or a rich knowledge base. It needs knowledge about the user and his understanding of the domain in order to produce a relevant and coherent discourse. We describe MENO, a prototype tutor for elementary PASCAL, which uses a set of speech patterns modelled after complex human discourse and a richly annotated knowledge base to produce a flexible interactive system for the user.

An important learning skill is the ability to make abstractions, i.e., to construct classification schemes which highlight similarities and differences. In this paper we shall outline the content of a undergraduate course which attempts to teach this skill in the context of teaching introductory LISP programming and problem solving. The key to this enterprise has been the development of: