In Peer Instruction (PI) an instructor displays a challenging multiple-choice question during lecture that students answer individually, discuss verbally with nearby peers, answer individually again, and finally, the instructor leads a discussion of the question. Peer Instruction typically increases student learning and motivation over traditional lecture. We added a text-chat mode to improve PI for remote synchronous learning. This feature assigns students to discussion groups to maximize the number of groups that have members with different answers. The tool was pilot tested in Winter 2022 and revised. In Fall 2022 and Winter 2023, it was tested at one institution. In Fall 2024, it was tested at four institutions. We conducted a log file analysis of student data from 1394 students and analyzed surveys with 848 student responses. We found that questions answered using the text-chat had a significantly higher improvement than those using traditional verbal discussion, although the two modes were tested with different questions. Interestingly, most of the students preferred to discuss the question verbally, although some preferred the text-chat discussion. These results inform efforts to improve the effectiveness of Peer Instruction and increase its adoption.

Despite decades of research on the effectiveness of Peer Instruction (PI), it can be hard to convince computing instructors to try it. In Peer Instruction an instructor displays a hard multiple-choice question during lecture, students answer individually, discuss with peers, and answer independently again. The main reasons computing instructors give for not adopting PI is a lack of awareness, concerns that it would result in less coverage of material in lecture, and the time needed to adopt it. We tried to increase awareness of PI and address instructors' concerns during a three-day summer instructor workshop with 14 instructors in 2024. A pre-survey was administered on the first day of the workshop and a post-survey on the last day. We administered a second post-survey in the fall of 2024. Later semi-structured interviews were conducted with nine instructors. The adoption rate from the 2024 workshop was over 50%, which was much higher than the previous two workshop's adoption rate of 23%. The workshop also increased knowledge of PI and alleviated several concerns. However, instructors wanted more follow-up support, an easier and faster way to create new PI questions, and better integration with their Learning Management System (LMS). Instructors who adopted the free tool, Peer+, reported positive student reactions and better understanding of student misconceptions. All instructors who adopted Peer+ plan to continue to use it.

Peer Reviewed

Despite evidence of its effectiveness, Peer Instruction (PI) has not been widely adopted by undergraduate computing instructors. In PI, an instructor displays a hard multiple-choice question that students answer individually, then discuss their answer with peers, then answer again, and finally an instructor leads a discussion of the question. Even though the benefits of PI are well documented, it can be difficult to convince computing instructors to move away from passive lectures. Major reasons why instructors do not adopt PI include a lack of awareness, lack of time, and concerns over their ability to cover content. We hypothesized that we could encourage the adoption of PI by creating Peer+, a free tool in an ebook platform, a searchable question bank, and running summer instructor workshops. We offered a three-day in-person summer workshop to a total of 37 instructors in 2022 and 2023. Instructors completed a pre-survey, immediate post-survey, and a follow-up post survey after the fall semester. We also conducted semi-structured interviews with 17 instructors. On the immediate post-survey most (33/37, 89%) instructors reported that they were very likely or likely to use the tool in the fall. However, on the follow-up survey, less than a quarter (6/26, 23%) actually did. The number one reason for not using the tool was a lack of time (18/26, 69%). Notably, all of the instructors who used Peer+ planned to use it again. This work informs efforts to increase the adoption of evidence-based pedagogical approaches in computing.

As generative AI products could generate code and assist students with programming learning seamlessly, integrating AI into programming education contexts has driven much attention. However, one emerging concern is that students might get answers without learning from the LLM-generated content. In this work, we deployed the LLM-powered personalized Parsons puzzles as scaffolding to write-code practice in a Python learning classroom (PC condition) and conducted an 80-minute randomized between-subjects study. Both conditions received the same practice problems. The only difference was that when requesting help, the control condition showed students a complete solution (CC condition), simulating the most traditional LLM output. Results indicated that students who received personalized Parsons puzzles as scaffolding engaged in practicing significantly longer than those who received complete solutions when struggling.

Dynamic programming (DP) is one of the most challenging topics in algorithms courses. Although there exist animation tools that assist with the understanding of DP algorithms, few existing tools are aimed at scaffolding the process of solving DP algorithm design problems. To help create a learning tool able to provide the affordances learners need when attempting DP problems, we analyzed learners' scratch paper to understand how learners approach DP problems. Based on scratch paper from 18 learners solving DP problems during a think-aloud study, we created a codebook that characterized different elements and methods used by the learners on their scratch paper. We found that learners had distinct preferences when attempting DP problems. Some learners preferred using example input with specific values to simulate ideal program executions, while some used math representations of example inputs to help derive formulas. Learners interacted with their example input in multiple ways, including filling in hand-drawn tables to organize the calculation process and dynamically interacting with the inputs by crossing, circling, or using arrows to visualize the relationships between inputs. These findings suggest potential interactions that need to be taken into consideration when designing tools to support learners in solving DP problems.

This paper investigates using micro Parsons problems as a novel practice approach for learning Structured Query Language (SQL). In micro Parsons problems learners arrange predefined code fragments to form a SQL statement instead of typing the code. SQL is a standard language for working with relational databases. Targeting beginner-level SQL statements, we evaluated the efficacy of micro Parsons problems with block-based feedback and execution-based feedback compared to traditional text-entry problems. To delve into learners’ experiences and preferences for the three problem types, we conducted a within-subjects think-aloud study with 12 participants. We found that learners reported very different preferences. Factors they considered included perceived learning, task authenticity, and prior knowledge. Next, we conducted two between-subjects classroom studies to evaluate the effectiveness of micro Parsons problems with different feedback types versus text-entry problems for SQL practice. We found that learners who practiced by solving Parsons problems with block-based feedback had a significantly higher learning gain than those who practiced with traditional text-entry problems.

The rapid advancement of AI-generated content (AIGC) promises to transform various aspects of human life significantly. This work particularly focuses on the potential of AIGC to revolutionize image creation, such as photography and self-expression. We introduce ContextCam, a novel human-AI image co-creation system that integrates context awareness with mainstream AIGC technologies like Stable Diffusion. ContextCam provides user’s image creation process with inspiration by extracting relevant contextual data, and leverages Large Language Model-based (LLM) multi-agents to co-create images with the user. A study with 16 participants and 136 scenarios revealed that ContextCam was well-received, showcasing personalized and diverse outputs as well as interesting user behavior patterns. Participants provided positive feedback on their engagement and enjoyment when using ContextCam, and acknowledged its ability to inspire creativity.

The capability of large language models (LLMs) to generate, debug, and explain code has sparked the interest of researchers and educators in undergraduate programming, with many anticipating their transformative potential in programming education. However, decisions about why and how to use LLMs in programming education may involve more than just the assessment of an LLM’s technical capabilities. Using the social shaping of technology theory as a guiding framework, our study explores how students’ social perceptions influence their own LLM usage. We then examine the correlation of self-reported LLM usage with students’ self-efficacy and midterm performances in an undergraduate programming course. Triangulating data from an anonymous end-of-course student survey (n = 158), a mid-course self-efficacy survey (n=158), student interviews (n = 10), self-reported LLM usage on homework, and midterm performances, we discovered that students’ use of LLMs was associated with their expectations for their future careers and their perceptions of peer usage. Additionally, early self-reported LLM usage in our context correlated with lower self-efficacy and lower midterm scores, while students’ perceived over-reliance on LLMs, rather than their usage itself, correlated with decreased self-efficacy later in the course.

Background: In CS1 courses, Parsons problems are a popular activity in which students are given blocks of code and asked to rearrange them into the correct order. Parsons problems often include incorrect blocks of code referred to as distractor blocks. Despite their widespread use, there have been few investigations into how distractor blocks impact student learning. Objectives: Our goals are to understand (1) the impact that including distractor blocks in Parsons problems has on learning and (2) the causality underlying that learning, if any. Methods: In this paper, we present the results of an explanatory sequential mixed methods study investigating the impact of distractor blocks on student learning. For the initial, quantitative stage, we use a randomized control trial to quantify the learning outcomes from practice with Parsons problems that include distractor blocks, as measured via post-test taken immediately after the practice activity and a retention test taken a week later. This study is followed by think-aloud interviews with 10 students practicing using a mix of Parsons problems that do and do not contain distractors to understand differences in how students approach those problems. Findings: Our findings show that students who practiced using Parsons problems that contained distractors performed 11 percentage points better on the immediate post-test (statistically significant) and 10 percentage points better on the retention test (approaching significance). The results of the think-aloud interviews indicate that grouping distractors with blocks of correct code causes students to more closely attend to the details of the code within those blocks. Implications: The results of this study indicate that distractors are essential when Parsons problems are used in a formative context. When they are not included, students may be able to successfully place blocks of code without attending to details of the code. This in turn limits their ability to learn new concepts or reinforce existing knowledge from those code blocks.

K-12 computing education research is a rapidly growing field of research, both driven by and driving the implementation of computing as a school and extra-curricular subject globally. Within discipline-based education research, it is a new and emerging field, drawing on fields such as mathematics and science education research for inspiration and theoretical bases. The urgency around investigating effective teaching and learning in computing in school alongside broadening participation has led to much of the field being focused on empirical research. Less attention has been paid to the underlying philosophical assumptions informing the discipline, which might include a critical examination of the rationale for K-12 computing education, its goals and perspectives, and associated inherent values and beliefs. The goals of this research project are to understand the implicit and hidden values, perspectives and goals underpinning computing education at school. This will be achieved through a critical examination of a wide body of literature leading to the development of a categorization and framework.

Parsons problems are a type of programming activity that present learners with blocks of existing code and requiring them to arrange those blocks to form a program rather than write the code from scratch. Micro Parsons problems extend this concept by having students assemble segments of code to form a single line of code rather than an entire program. Recent investigations into micro Parsons problems have primarily focused on supporting learners leaving open the question of micro Parsons efficacy as an exam item and how students perceive it when preparing for exams.To fill this gap, we included a variety of micro Parsons problems on four exams in an introductory programming course taught in Python. We use Item Response Theory to investigate the difficulty of the micro Parsons problems as well as the ability of the questions to differentiate between high and low ability students. We then compare these results to results for related questions where students are asked to write a single line of code from scratch. Finally, we conduct a thematic analysis of the survey responses to investigate how students' perceptions of micro Parsons both when practicing for exams and as they appear on exams.

Learning to program can be challenging, and providing high-quality and timely support at scale is hard. Generative AI and its products, like ChatGPT, can create a solution for most intro-level programming problems. However, students might use these tools to just generate code for them, resulting in reduced engagement and limited learning. In this paper, we present CodeTailor, a system that leverages a large language model (LLM) to provide personalized help to students while still encouraging cognitive engagement. CodeTailor provides a personalized Parsons puzzle to support struggling students. In a Parsons puzzle, students place mixed-up code blocks in the correct order to solve a problem. A technical evaluation with previous incorrect student code snippets demonstrated that CodeTailor could deliver high-quality (correct, personalized, and concise) Parsons puzzles based on their incorrect code. We conducted a within-subjects study with 18 novice programmers. Participants perceived CodeTailor as more engaging than just receiving an LLM-generated solution (the baseline condition). In addition, participants applied more supported elements from the scaffolded practice to the posttest when using CodeTailor than baseline. Overall, most participants preferred using CodeTailor versus just receiving the LLM-generated code for learning. Qualitative observations and interviews also provided evidence for the benefits of CodeTailor, including thinking more about solution construction, fostering continuity in learning, promoting reflection, and boosting confidence. We suggest future design ideas to facilitate active learning opportunities with generative AI techniques.

Peer Instruction (PI) is a learning activity that lets students solve a difficult multiple-choice question individually, submit their answer, discuss with peers to solve the problem collaboratively, and then submit the answer again. Despite plentiful evidence to support its effectiveness, PI has not been widely adopted by undergraduate computing instructors due to low awareness of PI, the effort needed to create PI questions, the limited instructional time needed for PI activities during lectures, and potential adverse reactions from students.

Parsons problems have been used to provide scaffolding for introductory learners. Instead of asking learners to write from scratch, Parsons problems provide blocks of mixed-up code, and ask learners to rearrange them into the correct order. In traditional Parsons problems, each block would contain one or more lines of code. While traditional Parsons problems have been widely used, there is an untapped potential to adapt them to practice to write a single line of code.

Regular expressions (regex) are a text processing method widely used in data analysis, web scraping, and input validation. However, students find regular expressions difficult to create since they use a terse language of characters. Parsons problems can be a more efficient way to practice programming than typing the equivalent code with similar learning gains. In traditional Parsons problems, learners place mixed-up fragments with one or more lines in each fragment in order to solve a problem. To investigate learning regex with Parsons problems, we introduce micro Parsons problems, in which learners assemble fragments in a single line. We conducted both a think-aloud study and a large-scale between-subjects field study to evaluate this new approach. The think-aloud study provided insights into learners' perceptions of the advantages and disadvantages of solving micro Parsons problems versus traditional text-entry problems, student preferences, and revealed design considerations for micro Parsons problems. The between-subjects field study of 3,752 participants compared micro Parsons problems with text-entry problems as an optional assignment in a MOOC. The dropout rate for the micro Parsons condition was significantly lower than the text-entry condition. No significant difference was found for the learning gain on questions testing comprehensive regex skills between the two conditions, but the micro Parsons group had a significantly higher learning gain on multiple choice questions which tested understanding of regex characters.

Gaze-only input techniques in VR face the challenge of avoiding false triggering due to continuous eye tracking while maintaining interaction performance. In this paper, we proposed GazeDock, a technique for enabling fast and robust gaze-based menu selection in VR. GazeDock features a view-fixed peripheral menu layout that automatically triggers appearing and selection when the user’s gaze approaches and leaves the menu zone, thus facilitating interaction speed and minimizing the false triggering rate. We built a dataset of 12 participants’ natural gaze movements in typical VR applications. By analyzing their gaze movement patterns, we designed the menu UI personalization and optimized selection detection algorithm of GazeDock. We also examined users’ gaze selection precision for targets on the peripheral menu and found that 4–8 menu items yield the highest throughput when considering both speed and accuracy. Finally, we validated the usability of GazeDock in a VR navigation game that contains both scene exploration and menu selection. Results showed that GazeDock achieved an average selection time of 471ms and a false triggering rate of 3.6%. And it received higher user preference ratings compared with dwell-based and pursuit-based techniques.

Learning to write is challenging for blind and low vision (BLV) people because of the lack of visual feedback. Regardless of the drastic advancement of digital technology, handwriting is still an essential part of daily life. Although tools designed for teaching BLV to write exist, many are expensive and require the help of sighted teachers. We propose LightWrite, a low-cost, easy-to-access smartphone application that uses voice-based descriptive instruction and feedback to teach BLV users to write English lowercase letters and Arabian digits in a specifically designed font. A two-stage study with 15 BLV users with little prior writing knowledge shows that LightWrite can successfully teach users to learn handwriting characters in an average of 1.09 minutes for each letter. After initial training and 20-minute daily practice for 5 days, participants were able to write an average of 19.9 out of 26 letters that are recognizable by sighted raters.

When teams of data scientists collaborate on computational notebooks, their discussions often contain valuable insight into their design decisions. These discussions not only explain analysis in the current notebook but also alternative paths, which are often poorly documented. However, these discussions are disconnected from the notebooks for which they could provide valuable context. We propose Callisto, an extension to computational notebooks that captures and stores contextual links between discussion messages and notebook elements with minimal effort from users. Callisto allows notebook readers to better understand the current notebook content and the overall problem-solving process that led to it, by making it possible to browse the discussions and code history relevant to any part of the notebook. This is particularly helpful for onboarding new notebook collaborators to avoid misinterpretations and duplicated work, as we found in a two-stage evaluation with 32 data science students.