Understanding unfamiliar code often requires insight into the original developer’s intentions, design rationale, and on-the-fly problem-solving strategies. Traditional version-control systems capture only coarse commits, leaving web foraging and developer notes disconnected. We present DevTales, a VSCode extension that integrates fine-grained subgoal labels, captured web-foraging activity, and LLM-generated narrative summaries directly with code in the IDE. We evaluated DevTales in three mixed-methods user studies spanning debugging, understanding rationale, and code reuse. Results show that subgoal labels and “See in Code” links enable rapid top-down navigation and code localization, while embedded web resources clarify unfamiliar artifacts. Narrative summaries proved valuable for answering deep “why” questions by explicitly linking causal relationships among history items, but were avoided during quick sensemaking contexts due to their verbosity. We discuss design implications for history-aware code search, adaptive narrative summarization, and inference transparency in LLM-generated stories.

Software developers have difficulty understanding the rationale and intent behind original developers’ design decisions. Code histories aim to provide richer contexts for code changes over time, but can introduce a large amount of information to the already cognitively demanding task of code comprehension. Storytelling has shown benefits in communicating complex, time-dependent information, yet programmers are reluctant to write stories for their code changes. We explored the utility of narratives made by generative AI. We conducted a within-subjects study comparing the performance of 16 programmers when recalling code history information from a list-view format versus a comparable AI-generated narrative format. Our study found that when using the story-view, participants were 16% more successful at recalling code history information, and had 30% less error when assessing the correctness of their responses. We did not find any significant differences in programmer’s perceived mental effort or their attitudes towards reuse when using narrative code stories.

Today, code reuse is typically limited to programmers reusing code that does exactly what they want to do, through APIs and modules, for example. Yet, the space of existing programs that could be reused is broader. In this poster, we consider reuse through re-purposing tasks in which programmers modify an existing codebase to implement a related task. Because this process requires that users build a relationship between the source code and the target program, we examine it using analogical reasoning. To understand the process programmers take and the barriers they encounter when performing reuse through re-purposing tasks, we conducted an exploratory study involving sixteen participants completing two tasks. We find that programmers have difficulty mapping analogies related to the underlying logic of source code, but easily identify visual analogies like interface structure. We discuss how information about a code's history can be useful during reuse, and in identifying logical analogies.

Undergraduate Teaching Assistants(TAs) in Computer Science courses are often the first and only point of contact when a student gets stuck on a programming problem. But these TAs are often relative beginners themselves, both in programming and in teaching. In this paper, we examine the impact of availability of corrected code on TAs’ ability to find, fix, and address bugs in student code. We found that seeing a corrected version of the student code helps TAs debug code 29% faster, and write more accurate and complete student-facing explanations of the bugs (30% more likely to correctly address a given bug). We also observed that TAs do not generally struggle with the conceptual understanding of the underlying material. Rather, their difficulties seem more related to issues with working memory, attention, and overall high cognitive load.

Teaching assistants (TAs) play a crucial role in Computer Science courses. When a student is stuck or confused, they often rely on a TA to help them understand a concept or debug their program. At the same time, TAs in Computer Science courses are often very new at teaching, and somewhat new at programming. They may lack the knowledge and resources necessary to help students learn effectively. This work seeks to better understand the nature of TA-student interactions and identify potential opportunities for improvement. We conducted an observational study of one-on-one TA-Student interactions during office hours of a Computer Science course, and analyzed these interactions through the lens of known practices of effective one-on-one tutors. We found that TA-Student interactions focus on code over concepts, and this focus may be detrimental to TAs’ use of good tutoring practices.

Code puzzles are a popular way to introduce young learners to computer programming. Managing cognitive load can lead to more effective learning, but it is unknown what levels of cognitive load should be maintained for successful learning in code puzzle systems. In this paper, we find that successful learners follow a trend of lower cognitive load, and we suggest guiding users towards this trend before introducing more difficult tasks. We then quantify puzzle-solving experiences into 7 principal components and explain their relationship with cognitive load and how they may be able to be used to improve the learning experience.

Modern programming frequently requires the use of APIs (Application Programming Interfaces). Yet many programmers struggle when trying to learn APIs. We ran an exploratory study in which we observed participants performing an API learning task. We analyze their processes using a proposed model of API learning, grounded in Cognitive Load Theory, Information Foraging Theory, and External Memory research. The results provide support for the model of API Learning and add new insights into the form and usage of external memory while learning APIs. Programmers quickly curated a set of API resources through Information Foraging which served as external memory and then primarily referred to these resources to meet information needs while coding.

Code Puzzles can be an engaging way to learn programming concepts, but getting stuck in a puzzle can be discouraging when no help or feedback is available. Teachers and facilitators can alleviate this problem in a classroom setting, but it can be hard for teachers to keep track of who needs help and who is likely to resolve their problem on their own, especially in a large classroom. This work is a step toward helping teachers optimize their time by automatically gauging which students may benefit from an intervention at any given time. We use information about the bugs present in student code to predict which students are more likely to abandon the puzzle or take too long in solving it. Ultimately, we envision that teachers could use these predictions to make decisions about whom they should help next, and how.

Code puzzles are an increasingly popular way to introduce youth to programming. Yet our knowledge about how to maximize learning from puzzles is incomplete. We conducted a data collection study and trained a model that predicts cognitive load, the mental effort necessary to complete a task, on a future puzzle. Controlling cognitive load can lead to more effective learning. Our model suggests that it is possible to predict Cognitive Load on future problems; the model could correctly distinguish the more difficult puzzle within a pair 71%-79% of the time. Further, studying the model itself provides new insights into the sources of puzzle difficulty, the factors that contribute to Cognitive Load, and their inter-relationships. Finally, the ability to predict Cognitive Load on a future puzzle is an important step towards the creation of adaptive code puzzle systems.

Though support for learning computing in schools is growing, many children still begin learning to program without formal support in open-ended programming environments. While researchers have evaluated the final code of these types of projects, we know little about how users’ behaviors and usage of support tools relate to understanding. We ran a study where participants had open-ended programming time with access to one of two support tools: suggestions or tutorials. Participants then completed four tasks which required understanding of the suggestion or tutorial content. We did not find an effect of suggestions compared to tutorials on knowledge application, but we did find that many participants who performed better tended to explore more of the interface, code behaviors, and support tools. Our results suggest that future tools for encouraging learning during open-ended programming should likely focus on supporting users who tend to explore less on their own.

In today’s world, learning new APIs (Application Programming Interfaces) is fundamental to being a programmer. Prior research suggests that programmers learn on-the-fly while they work on other project-related tasks. Yet, this process is often inefficient. This inefficiency has inspired research seeking to understand and improve API learnability. While the existing research has provided insight into API learning, we still have a fractured understanding of the process of learning a new API. In this paper, we take the first steps towards developing a theoretical model of API learning by combining predictions from Information Foraging Theory (IFT) to describe information search behavior, Cognitive Load Theory (CLT) to describe learning, and External Memory (EM) to describe how API learners augment their short term memories. Our proposed model is consistent with existing research on barriers to learning APIs and helps to provide explanations for these barriers as well as suggest new research directions.

We analyze existing data of students completing coding puzzles through the lens of a debugging process, in order to study the impact of different types of errors that students make as they solve the puzzle. We develop a scheme for categorizing the errors present in the student code at any given time, and use it to create a taxonomy of the trajectories that students take to arrive at the correct solution. We find that these metrics are expressive enough to capture important distinguishing characteristics of students' puzzle-solving strategies.

APIs (Application Programming Interfaces) and code libraries have become highly integrated into the programming process. They allow programmers to reuse large segments of functionalities. However, as free and often open-source commodities, the support for programmers to learn how to use these valuable resources is not always complete. Researchers have repeatedly found that API learning is a highly problematic process with many barriers. However, much of the work on the difficulties using and learning APIs has relied on retrospective descriptions of the process or questions programmers post on forums. Furthermore, these explorations of difficulties in learning APIs have not taken into account theories about learning or information foraging. In this works-in-progress poster, we present an early evaluation of a model that describes API learning using both information foraging and cognitive load theory.

Independent novice programmers in open-ended contexts rely on help systems to support their learning. These help systems are often laboriously hand-authored by experts. This paper describes a semi-automatic process for the creation of a suggestion-based help system. We demonstrate and evaluate the potential utility of our approach within a blocks-based programming environment for children. With less human effort per suggestion, our approach generated a set of suggestions comparable to a hand-authored set and a set of original suggestions. We ran a study to explore the number and types of suggestions children received, accessed, and used. In 30 minutes, children on average received 9 suggestions, accessed 2.6 suggestions, and inserted 0.8 new concepts from suggestions.

Programmers, especially novices, often have difficulty learning new APIs (Application Programming Interfaces). Existing research has not fully addressed novice programmers' unawareness of all available API methods. To help novices discover new and appropriate uses for API methods, we designed a system called the Example Guru. The Example Guru suggests context-relevant API methods based on each programmer's code. The suggestions provide contrasting examples to demonstrate how to use the API methods. To evaluate the effectiveness of the Example Guru, we ran a study comparing novice programmers' use of the Example Guru and documentation-inspired API information. We found that twice as many participants accessed the Example Guru suggestions compared to documentation and that participants used more than twice as many new API methods after accessing suggestions than documentation.

There are a variety of learning resources with the potential to support children in learning programming independently. While many of them have been evaluated in laboratory settings, we know little about how children choose to use these resources on their own. We conducted a study organized around a film festival to explore children's open-ended use of four different learning supports: tutorials, code puzzles, in-application documentation and code suggestions. The study began with a workshop to introduce the programming environment and available tools, continued through two weeks of home use, and culminated in a film festival. Results suggest that participants leveraged in-context forms of help most frequently, but valued documentation for question-answering and suggestions for opportunistic learning.

Programmers of all experience levels attempt to leverage code snippets with varying success, often as reminders or to learn new skills. To date, little work has explored the specific elements within code snippets that are challenging for novices. Comparing how novices and experts recall code snippets may expose what code elements programmers focus on and inform new approaches for improving examples for inexperienced programmers. We conducted a study, inspired by past novice-expert studies, in which we asked everyday, occasional, and non-programmers to study and then recall code snippets. The key distinctions and similarities in the types and locations of recalled tokens provide insight for a set of recommendations that could improve the presentation of code snippets.

The frequency of programmers attempting to use code examples has prompted significant research on code examples for text languages. Yet, few systems address issues in novice use of examples in blocks programming languages. Research has begun to explore the difficulties novices have using examples in blocks programming languages. This work addresses one such issue: novices often do not notice or focus on the important elements in examples. This work-in-progress poster presents lessons learned on how to design examples that help novices notice critical elements.

Parsons problems are an increasingly popular method for helping inexperienced programmers improve their programming skills. In Parsons problems, learners are given a set of programming statements that they must assemble into the correct order. Parsons problems commonly use distractors, extra statements that are not part of the solution. Yet, little is known about the effect distractors have on a learner's ability to acquire new programming skills. We present a study comparing the effectiveness of learning programming from Parsons problems with and without distractors. The results suggest that distractors decrease learning efficiency. We found that distractor participants showed no difference in transfer task performance compared to those without distractors. However, the distractors increased learners cognitive load, decreased their success at completing Parsons problems by 26%, and increased learners' time on task by 14%.

Tutorials and code puzzles are commonly used in today's novice programming environments to introduce computer programming to children. While research has explored the effectiveness of each instructional format at teaching different kinds of information independently, little work has explored learners' perceptions of value in each or the strategic decisions users make around the instructional format when learning to program. We present a study in which learners selected from a set of tutorials and puzzles with an identical set of programming content. We explore the reasoning behind their choices and the potential implications for the learning support available in future programming environments.

Many novice programmers use blocks-based programming environments outside of classrooms, due to a lack of computer science education in schools. Many solutions for supporting learning in these environments are out of the context of the programming environment and the user's project. In this poster, we present a way of expanding novice users' knowledge of a blocks-based programming environment by suggesting examples during open-ended programming.

Looking Glass is the successor to Storytelling Alice designed for middle and high school students. By dragging and dropping, users can construct programs that direct the behavior of characters in a 3D scene. The system consists of a downloadable application and an online community.

Many novice programming environments use puzzle-like approaches to help novice programmers acquire new programming skills independently. Yet, little is known about 1) how puzzles can support effective learning of programming skills and 2) how learning programming using a puzzle-based approach compares to more a traditional tutorial style approach. We conducted a pair of studies to explore these two questions. First, we report lessons learned on the design of programming completion puzzles, their interface within a novice programming environment, and the design of a puzzle curriculum drawn from our first, formative study. We then report on a second study that compared the learning effectiveness of programming puzzles and tutorials. The results suggest that puzzles are a promising approach for introducing programming concepts within novice programming environments. Puzzle users performed 26% better on transfer tasks compared to tutorial users, while taking 23% less time to complete the learning materials.

Both experienced and novice programmers use examples while programming, whether from tutorials, forums, or source code. Novice programmers, however, often find it challenging to use unfamiliar example code. Little is known about the challenges of using examples, making it difficult to design support for novice programmer example use. We ran an exploratory study of novices using examples to complete programming tasks. To analyze programming behaviors, we define the `realization point' as the time when the participants discover the crucial concept in an example. Our results show that participants spent more time after the realization point using the example than they did identifying which part of the example to use. We describe hurdles and strategies, types of tasks behaviors, and finally, implications for supporting example use.

Stroke survivors’ unsupervised therapeutic exercise motions are often accompanied by harmful compensatory motions that prevent proper motor recovery and introduce additional health issues. These compensatory motions are often performed unconsciously and are difficult to prevent. Motion-based games show promise for motivating patients to perform stroke rehabilitation exercises at home by themselves. Currently, exercises with these games are likely to contain undesired compensatory motions. In this article, we provide the design and empirical evaluation of a motion-based game system that addresses the issue of compensation in therapeutic games. We introduce a technique to identify and measure compensation, develop a game that meaningfully uses exercise and compensation as inputs, and use incentives and disincentives to reduce compensation. We show that this technique outperforms existing approaches by significantly reducing compensatory motions during therapeutic exercise. This has important implications for therapeutic games, which can use our findings to improve the quality of motions to be closer to therapist-supervised motions. Our techniques can increase the effectiveness of therapeutic games and reduce the possibility that they may cause harm in long-term use.

One way to provide feedback to independent novice programmers is by leveraging experienced programmers as code reviewers. To provide this feedback at a large scale, experienced programmers can author heuristic programs, or rules, that automatically determine whether a novice program should receive certain feedback. This work presents the lessons learned from designing a tool to enable rule authoring.

Enabling middle school children to learn from code shared on the internet may provide computer science learning opportunities to those who would not otherwise have them. We augmented a programming environment designed for middle school children to automatically generate tutorials from code snippets in order to help users learn new programming skills. In our new system, users select code snippets from a program shared on the web and then complete an automatically generated tutorial in order to re-create that snippet within their own program. To evaluate the potential learning gains from our generated tutorials, we conducted a between-subjects study in which we evaluated the performance of children introduced to new programming constructs through automatically generated tutorials. Participants who used the automatically generated tutorials performed 64% better on a near transfer task compared to participants without generated tutorials.

Novice programmers may lack the experience to recognize opportunities to either improve their code or apply unfamiliar programming constructs. Yet, these opportunities are often clear to an experienced programmer. In this paper, we describe an exploratory study investigating 1) the potential value of the suggestions experienced programmers make to novice programmers and 2) the ways experienced programmers envision identifying other programs that would benefit from the same suggestion. The results of our study suggest that experienced programmers make suggestions that can introduce new programming constructs to novice programmers. The participants in our study most commonly made suggestions that improve the code quality of novice programs, rather than changing their output. Furthermore, experienced programmers could often state a simple heuristic rule to use in identifying other novice programs that would benefit from their suggestion. Participants were able to author the rules in pseudocode, mostly using combinations of iteration and comparison to find patterns of problematic code. However, based on a test implementation of a selected set of rules for these suggestions, we conclude that support for improving rules through review and community input will be valuable.

Research suggests that storytelling can motivate middle school students to explore computer programming. However, difficulties finding and realizing story ideas can decrease time actually spent on programming. In this paper, we present guidelines for constructing story scenes that reliably inspire ideas for novice programmers creating stories. To evaluate the impact of pre-built scenes with strategic design constraints on early programming behavior and attitudes, we conducted a between-subjects study comparing participants who used pre-built scenes and participants who crafted their own scenes. The results suggest that story starter scenes enable novice users to explore programming in the environment sooner, allow users to add and modify significantly more novel programming constructs during the length of the study, and maintain motivation for learning to program via storytelling.

To facilitate long-term engagement in programming for middle school children, we developed the Looking Glass Community. The Community includes both a website and integrated access to community resources within the novice programming environment, Looking Glass. We discuss how we designed the Community to support engagement by providing a source for initial ideas, support for learning new skills, positive feedback, and role models.

Stroke is the leading cause of long-term disability among adults in industrialized nations; approximately 80% of people who survive a stroke experience motor disabilities. Recovery requires hundreds of daily repetitions of therapeutic exercises, often without therapist supervision. When performing therapy alone, people with limited motion often compensate for the lack of motion in one joint by moving another one. This compensation can impede the recovery progress and create new health problems. In this work we contribute (1) a methodology to reliably sense compensatory torso motion in the context of shoulder exercises done by persons with stroke and (2) the design and experimental evaluation of operant-conditioning-based strategies for games that aim to reduce compensatory torso motion. Our results show that these strategies significantly reduce compensatory motions compared to alternatives.

To support children learning to use new software applications independently, tutorial systems should prevent errors and ensure that users are able to transfer tutorial skills to a new context effectively. In this paper, we describe the formative development and evaluation of on-request stencils, an interaction technique that both prevents children from making errors within a tutorial and significantly improves their ability to transfer tutorial skills to a related task. Using on-request stencils, users can attempt a task independently. If they encounter difficulty, users can request step by step tutorial overlays to guide them through the current task. In a study comparing tutorial performance, task performance, and attitudes, we found that users of on-request stencils successfully completed 47% more transfer tasks than users of persistent stencils. There were no significant differences between the two groups in tutorial performance or attitudes towards the software system.

The web holds an abundance of source code examples with the potential to become learning resources for any end-user. However, for some end-users these examples may be unusable. An example is unusable if a user cannot select the code in the example that corresponds to their interests. Research suggests that non-programmers struggle to correctly select the code responsible for interesting output functionality. In this paper we present Dinah: an interface to support non-programmers with selecting code causing graphical output. Dinah assists non-programmers by providing concurrency support and in-context affordances for statement replay and temporally based navigation.

The wealth of code available through the web has the potential to dramatically change the way we learn to program. This includes inexperienced programmers, who may struggle to find code in example programs that relate to observable program features. We present a comparative study of three tools for assisting non-programmers with finding program code corresponding to a program's graphical output. From this study we also identify a model which captures the goals inherent in non-programmers' code search processes for this type of search task. Our results suggest a global pause marker may be an effective tool to support non-programmers' search.

In the United States alone, more than five million people are living with long term motor impairments caused by a stroke. Video game-based therapies show promise in helping people recover lost range of motion and motor control. While researchers have demonstrated the potential utility of game-based rehabilitation through controlled studies, relatively little work has explored longer-term home-based use of therapeutic games. We conducted a six-week home study with a 62 year old woman who was seventeen years post-stroke. She played therapeutic games for approximately one hour a day, five days a week. Over the six weeks, she recovered significant motor abilities, which is unexpected given the time since her stroke. Through observations and interviews, we present lessons learned about the barriers and opportunities that arise from long-term home-based use of therapeutic games.

Stroke is the leading cause of long term disability among adults in industrialized nations. The partial paralysis that stroke patients often experience can make independent living difficult or impossible. Research suggests that many of these patients could recover by performing hundreds of daily repetitions of motions with their affected limbs. Yet, only 31% of patients perform the exercises recommended by their therapists. Home-based stroke rehabilitation games may help motivate stroke patients to perform the necessary exercises to recover. In this paper, we describe a formative study in which we designed and user tested stroke rehabilitation games with both stroke patients and therapists. We describe the lessons we learned about what makes games useful from a therapeutic point of view.

We describe a code reuse tool for use in the Looking Glass IDE, the successor to Storytelling Alice [17], which enables middle school students with little to no programming experience to reuse functionality they find in programs written by others. Users (1) record a feature to reuse, (2) find code responsible for the feature, (3) abstract the code into a reusable Actionscript by describing object "roles," and (4) integrate the Actionscript into another program. An exploratory study with middle school students indicates they can successfully reuse code. Further, 36 of the 47 users appropriated new programming constructs through the process of reuse.

Source code on the Web is a widely available and potentially rich learning resource for non-programmers. However, unfamiliar code can be daunting to end-users without programming experience. This paper describes the results of an exploratory study in which we asked non-programmers to find and modify the code responsible for specific functionality within unfamiliar programs. We present two interacting models of how non-programmers approach this problem: the task process model and the landmark-mapping model. Using these models, we describe code search strategies non-programmers employed and the difficulties they encountered. Finally, we propose guidelines for future programming environments that support non-programmers in finding functionality in unfamiliar programs.

We describe Storytelling Alice, a programming environment that introduces middle school girls to computer programming as a means to the end of creating 3D animated stories. Storytelling Alice supports story creation by providing 1) a set of high-level animations, that support the use of social characters who can interact with one another, 2) a collection of 3D characters and scenery designed to spark story ideas, and 3) a tutorial that introduces users to writing Alice programs using story-based examples. In a study comparing girls' experiences learning to program using Storytelling Alice and a version of Alice without storytelling support (Generic Alice), we found that users of Storytelling Alice and Generic Alice were equally successful at learning basic programming constructs. Participants found Storytelling Alice and Generic Alice equally easy to use and entertaining. Users of Storytelling Alice were more motivated to program; they spent 42% more time programming, were more than 3 times as likely to sneak extra time to work on their programs, and expressed stronger interest in future use of Alice than users of Generic Alice.

Traditional approaches to teaching computer science are often unsuccessful in attracting girls into the discipline. Our hypothesis is that presenting computer programming as a means to the end of storytelling will help motivate girls to learn to program, a traditional gateway to computer science. In this paper, we present a case study in designing a version of the Alice programming system to support storytelling. We present lessons we learned about what supports are necessary to enable girls to program animated movies and describe the kinds of programming tasks that arise in girls' stories

Users of traditional tutorials and help systems often have difficulty finding the components described or pictured in the procedural instructions. Users also unintentionally miss steps, and perform actions that the documentation's authors did not intend, moving the application into an unknown state. We introduce Stencils, an interaction technique for presenting tutorials that uses translucent colored stencils containing holes that direct the user's attention to the correct interface component and prevent the user from interacting with other components. Sticky notes on the stencil's surface provide necessary tutorial material in the context of the application. In a user study comparing a Stencils-based and paper-based version of the same tutorial in Alice, a complex software application designed to teach introductory computer programming, we found that users of a Stencils-based tutorial were able complete the tutorial 26% faster, with fewer errors, and less reliance on human assistance. Users of the Stencils-based and paper-based tutorials attained statistically similar levels of learning.