This paper introduces ReflectSumm, a novel summarization dataset specifically designed for summarizing students’ reflective writing. The goal of ReflectSumm is to facilitate developing and evaluating novel summarization techniques tailored to real-world scenarios with little training data, with potential implications in the opinion summarization domain in general and the educational domain in particular. The dataset encompasses a diverse range of summarization tasks and includes comprehensive metadata, enabling the exploration of various research questions and supporting different applications. To showcase its utility, we conducted extensive evaluations using multiple state-of-the-art baselines. The results provide benchmarks for facilitating further research in this area.

This study explored the effectiveness of scaffolding in students' reflection writing process. We compared two sections of an introductory computer programming course$(\mathrm{N}=188)$. In Section 1, students did not receive any scaffolding while generating reflections, whereas, in Section 2, students were scaffolded during the reflection writing process. Student reflections were collected using two versions of the CourseMIRROR application (standard version in Section 1 and adaptive version in Section 2). By using Natural Language Processing (NLP) algorithms, the app calculated a reflection specificity score for each reflection. We conducted an independent sample t-test between the students' reflection specificity scores in these two sections. The results indicated that students using adaptive versions wrote more specific reflections than students using the standard version of the app, suggesting that scaffolding helped students write more specific reflections, which may be helpful in their overall learning outcomes in an introductory computer programming course.

This study explores the relationship between students' written and survey-based reflections in a first-year engineering class. We collected student reflections using the CourseMIRROR application from 395 students in an engineering class at a midwestern university. After each class during a semester, students were asked to generate a written reflection (in an open-ended format) and their perceived rating (in a Likert-style format) on the lecture's confusing or interesting aspects. We used Spearman correlation statistics to evaluate the relationship between the students' written reflection meta-data (i.e., specificity score and text length) and their perceived lecture rating as confusing or interesting. The results showed that the students tended to rate a lecture as very confusing when they wrote reflections highly relevant to prompts and lecture contents (i.e., reflection quality). Also, we found that the students rating a lecture as very confusing often write a relatively short reflection on the confusing question.

This work-in-progress research paper examines the relationship between two aspects of students' engagement and academic performance.With the boom of technology-mediated learning environments, many educational applications are integrated into STEM courses. However, the effectiveness of these applications in the learning environments is contingent upon factors including but not limited to applications' ease of use, relevance to courses, students' engagement with the application, and perceived value of the application in the context of students' learning. This work-in-progress paper uses two aspects of engagement in a mobile technology-mediated learning environment and explores their relationship with students' academic performances. The two perspectives of engagement include 1) students' engagement with the course – Academic Engagement and 2) students' engagement with the application used in the course – Application Engagement. We collected the data from 110 first-year engineering students enrolled in a required engineering class programming in MATLAB. Students self-reported their academic engagement on four dimensions: behavioral, emotional, social, and cognitive. In addition, students used a mobile application called CourseMIRROR. The application prompted students to write their reflections on each lecture throughout the semester, asking about its interesting or confusing points. The application uses Natural Language Processing (NLP) algorithm to create the summaries of these reflections. For application engagement, we used the number of times students viewed the summary through embedded data analytics in the CourseMIRROR application. For students' academic performance, we used the students' total scores in the course. We hypothesize that these two engagement perspectives are related to the students' academic performance. Specifically, the study will be guided by the following research questions: 1) To what degree do students' academic and application engagement relate to their academic performance? And 2) Do students with high engagement (i.e., academic or application) perform better in their exams? We analyzed the data using Pearson product-moment correlation and multiple regression to predict the students' academic performance and its relationship with students' academic engagement and application engagement. This work-in-progress paper presents the results of these analyses and their implications and provides future research directions.

We present the iterative design, prototype, and evaluation of CourseMIRROR (Mobile In-situ Reflections and Review with Optimized Rubrics), an intelligent mobile learning system that uses natural language processing (NLP) techniques to enhance instructor-student interactions in large classrooms. CourseMIRROR enables streamlined and scaffolded reflection prompts by: 1) reminding and collecting students' in-situ written reflections after each lecture; 2) continuously monitoring the quality of a student's reflection at composition time and generating helpful feedback to scaffold reflection writing; and 3) summarizing the reflections and presenting the most significant ones to both instructors and students. Through a combination of a 60-participant lab study and eight semester-long deployments involving 317 students, we found that the reflection and feedback cycle enabled by CourseMIRROR is beneficial to both instructors and students. Furthermore, the reflection quality feedback feature can encourage students to compose more specific and higher-quality reflections, and the algorithms in CourseMIRROR are both robust to cold start and scalable to STEM courses in diverse topics.

This study investigated the relationship between robotics team members' collaborative behaviors, communication and coordination in particular, during hypothetical challenges and their teams' actual performance during physical robotics challenges. Dataset included robot performance, robot design, research project, core values, and collaboration quality scores for 61 K-8 Robotics teams (N = 366) that participated in a FIRST LEGO League (FLL) Championship in 2015. Our analysis primarily focused on how well the collaboration quality scores predict team performance across different categories. Results indicated that the level of collaboration quality among team members is significantly associated with the team performance in the robotics tournament.

Wencan Luo, Xiangmin Fan, Muhsin Menekse, Jingtao Wang, Diane Litman. Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations. 2015.