The accessibility of interactive technologies is often evaluated using checklists that are low-level, numerous, and platform-specific. Such checklists are typically used by accessibility experts, leaving everyday designers and developers with little support for assessing their own interfaces. To make accessibility evaluations easier to conduct, we devised a set of nine “ability heuristics” that prompt designers to engage with accessibility throughout the design process. We empirically evaluated these ability heuristics with 37 design students, comparing them to usability heuristics and WCAG. The ability heuristics emphasized the quality of accessibility features compared to the other methods, and surfaced issues that were more broadly dispersed across disability groups. Further, the students found the heuristics were as easy to use as the alternative methods. We argue that the heuristics help to move beyond binary notions of accessibility, pushing designers to consider the quality of features across diverse disabilities and the range of abilities within.

Motivation. Asking for help is an essential skill in computing. However, many students face barriers to help-seeking, such as not being aware of help-seeking resources, doubting the usefulness of resources, fearing judgment from their peers, or not wanting to rely on others for help. Prior work on help-seeking in computing contexts has not directly considered how to include student perspectives in the design of help-seeking structures. Moreover, marginalized students face additional challenges due to inequities they face in computing. Therefore, it is imperative to elicit the voices of students marginalized in the community to address all challenges students face.

Background: Prior research argues that positive learning environments support student learning, belonging, and the desire to continue learning. In order to create joyful learning environments, we must first study what characterizes such learning spaces. However, prior research has not yet asked students what makes CS learning environments joyful. Objective: In this study, we sought to understand students’ joyful experiences in secondary CS classrooms and the elements of the learning environments that fostered those experiences. Method: We interviewed 8 students who had joyful CS experiences about those experiences, the environments in which they occured, and about the pedagogies their teachers used. We then interviewed 5 of their teachers to better understand the environments and pedagogies that cultivated joyful CS experiences. Findings: We found 4 elements in common in each student’s experience: teachers worked to relate to students, prioritize a relationship with learning, frame CS with purpose, and reify a joyful CS experience. Implications: These findings offer the opportunity for researchers and educators to reconsider pedagogies and design learning environments that engender joyful CS experiences for students. This will better enable students to build relationships with CS.

Background. Prior work has often speculated about learning synergies between computer science and language arts, building upon parallels between natural languages and programming languages and between writing and programming. Studies of primary classrooms have found applications of CS learning to reading literacy and applications of programming to interactive storytelling. Objectives. While prior work has richly explored youth learning, it has not explored opportunities in adolescent spaces. We sought to understand what additional synergies might be possible in relation to student culture and identity, particularly when CS and language arts are combined in a transdisciplinary way by leveraging programmable media that specifically centers text, language, culture, and writing. Our study examined what learning can occur and what structural factors in teaching, pedagogy, and programming media influenced this learning. Methods. We taught a 6-week, 20-hour transdisciplinary creative coding and writing course with seven 14-16 year old students using Wordplay, an educational programming language for creating multilingual interactive typography expressions. We captured student work samples, student reflections, and teacher reflections, and conducted an inductive thematic analysis on what tensions and opportunities arose between the two disciplines. Results. We found that students employed code to express their identity in ways that text alone could not capture, that disciplinary writing practices around outlining and planning mutually reinforced cross-disciplinary learning, that the richer identity expression encouraged peer learning, and that integration enabled writing-interested students to develop interests in CS, and CS-interested students to develop interest in writing. Conclusion. These findings suggest that creative writing and CS are synergistic not only for children, but for adolescents as well, particularly when programmable media centers on textual, typographic, and written form.

Students with disabilities face a variety of challenges in computer science education including those related to inaccessible curriculum, instruction and tools. In addition, few computer science classes teach students to understand accessibility and design accessible technology. This BOF will bring together individuals who are interested in increasing the accessibility of computing education as well as those interested in teaching about accessibility. Participants will share strategies to help each other do a better job of addressing accessibility in our classes and research projects. Resources related to accessible tools and instruction, universal design of learning, opportunities for students, teaching accessibility, and more will be shared.

Most software in the world is not accessible to people with disabilities, in part because most people who create software do not know how to make it accessible. Using the new free online book Teaching Accessible Computing, participants in this special session will learn how they can incorporate accessibility topics into their favorite CS courses, like introductory programming, data structures, or artificial intelligence courses. The editors of the book will lead the special session, and several authors of the book's 17 chapters will talk about their chapters and lead topic groups to go into more depth about how they incorporated accessibility into their CS courses. Participants will leave this session with increased enthusiasm and concrete resources to support teaching accessibility in their CS topic courses, establishing connections to the broader community of CS educators and researchers interested in teaching accessibility.

This Birds-of-a-Feather (BOF) session provides a space for LGBTQ+ educators, researchers, students, and allies to connect, share, and collaborate. It builds upon the strong foundation of previous successful BOFs focused on cultivating and sustaining the LGBTQ+ community (Kivuva et al., 2025) and the broad interest shown in panels on advancing LGBTQIA+ voices (DuBow et al., 2024). Recognizing the diverse needs within our community, this session is built around four concurrent, facilitated breakout groups, allowing attendees to self-select into the conversation most relevant to them. The importance of affinity spaces for ''gender-based community building'' and fostering a sense of belonging is a key aspect of broadening participation (Swackhamer et al., 2022). The session will begin with a collective welcome and introduction to ground us in a shared sense of community before self-selecting into roundtables. The four roundtable themes are: (1) An affinity space for sharing personal experiences from the past year; (2) Supporting LGBTQ+ students and building departmental community; (3) Advancing LGBTQ+-focused research in CS education; and (4) Professional development focused on grant writing and career advancement. This unique structure ensures that, whether attendees are seeking peer support, pedagogical strategies, research collaboration, or career mentorship, they will find a dedicated space. Participants will leave with new connections and targeted, actionable insights to support their personal and professional lives.

Figure 1: Cognimates interface showing coding blocks, AI chat, and image generation features.

Students with disabilities face a variety of challenges in computer science education including those related to stigma around disability, inaccessible curriculum, instruction and tools, disability disclosure, and a lack of mentors. In addition, few computer science classes teach students to understand accessibility and design accessible technology. This BOF will bring together individuals who are interested in increasing the representation of people with disabilities in computing and improving their success as well as those interested in teaching about accessibility. Participants will share strategies to help each other do a better job of addressing disability inclusion and accessibility in our classes and research projects. Resources related to accessible tools and instruction, universal design of learning, opportunities for students, teaching accessibility, and more will be shared.

This Birds-of-a-Feather (BoF) session offers LGBTQ+ researchers, educators, and practitioners in computing education an inclusive space for community-building, advocacy, and creative expression. The session seeks to expand beyond traditional networking by offering attendees a variety of opt-in activities, such as fiber arts, crafting, media sharing, and Pittsburgh-themed games. Attendees will have the opportunity to decompress after a long conference day, participate in discussions on queer scholarship and activism, and explore ways to make computing education more inclusive to LGBTQ+ individuals.

In the early 2000s, software development was shifting from offline to online, and from command line to IDE. We discuss our 2004 paper examining the impact of this shift on developers’ program comprehension behaviors, our motivation for the work, and it's impact on the last twenty years of empirical studies and developer tool innovations. We end with a discussion of the possible unintended impacts LLMs have on program comprehension in the coming decades.

Background: The growing complexity of the impacts of computing technologies on adolescents’ lives requires them to make similarly complex decisions around technology, fueling a rise in education efforts to look at the ethical implications of these advancements with young people. Though prior computing ethics education efforts integrate ethical perspectives, they have rarely drawn from scholarship on how to teach ethics and philosophy.

Objectives: Assessments play a crucial role in computer science courses by providing insights into student learning. While previous research has explored various aspects of assessments, little attention has been given to assessment policies that instructors devise and their impact on students’ experiences. Our goal was to investigate: How do assessment policies shape marginalized students’ experiences in coding classes?Method: We conducted 19 semi-structured interviews with post-secondary students currently enrolled in or completed a class where their code was evaluated. To recruit, we primarily targeted students from underrepresented racial groups in computer science. Many of these students attended large 4-year public universities. During the interviews, we inquired about students’ experience with different assessment policies and how those policies affected their lives and experiences completing the assignments. Results: Our findings revealed ten distinct ways policy and students’ lives interacted to create or heighten inequities, which significantly shaped marginalized students’ lives. Many policies did not consider the unique experiences of their students and students’ needs. Additionally, due to unclear and strict policies, students experienced frustration, confusion, and demotivation, consequently diminishing their sense of belonging in computer science and weakening their self-efficacy as programmers. This reveals the negative consequences of poor assessment policy choices and provides insight into how assessment policies can create barriers to learning computer science for marginalized students.

Many computer science education efforts promise liberation and equality, but that promise often goes unfulfilled. Teaching computation through E-textiles has been one way to achieve this promise because it has increased student engagement and enabled identity work. Although some approaches to teaching CS through E-textiles have been demonstrated as effective, there is not yet work using a programmable electronic embroidery machine (computational embroidery), or work that makes culture itself a topic of learning. In a six-week summer school course, we explored this opportunity, teaching a culture-centric embroidery class that combined hand embroidery and computational embroidery. Students incorporated their identities into their projects by using a block-based coding language to create embroidered patterns. They enthusiastically engaged with the programming aspects of the course and sought to make complicated and beautiful work that interwove their diverse cultures and identities. This paper offers insights into what it is like to teach computing with a cultural lens. Our curriculum and pedagogy offer instructors a template to incorporate these technologies and topics into their courses.

Neoliberal capitalism - the favoring of privatization, marketization, and deregulation of public resources - is hegemonic in education. It shapes our institutions and everyday life, including educational practices. The increasing push for market driven computing curriculum and large tech companies' involvement in research grants and educational programs are some examples of neoliberal influences in CS education. It is not only industry and industry-backed not-for-profits such as Code.org that reify neoliberal agendas, but in many cases, our very own communities, educational institutions, and curricula. Thus, a critical and reflective examination of our educational practices is necessary. In this Birds-of-a-Feather (BoF) session, we aim to conduct reflections and discussions on the role and influence of neoliberal capitalism in computing education. We will explore 1) the influence of deregulation, 2) public-private partnerships, 3) state-subsidized corporate wealth accumulation, and 4) centralization on pedagogy, research, and academic "success". We will discover reflective questions and practices that could help us critically examine our roles and responsibilities as computing educators and professionals. Eventually, we hope to begin a larger conversation and form a community to develop collective strategies and pedagogical ideologies to resist neoliberal influences, reimagine alternatives, and repair damages caused by neoliberal computing education.

Students with disabilities face a variety of challenges in computer science education including those related to stigma around disability, inaccessible curriculum, instruction and tools, disability disclosure, and a lack of mentors. In addition, few computer science classes teach students to understand accessibility and design accessible technology. This BOF will bring together individuals who are interested in increasing the representation of people with disabilities in computing and improving their success as well as those interested in teaching about accessibility. Participants will share strategies to help each other do a better job of addressing disability inclusion and accessibility in our classes and research projects. Resources related to accessible tools and instruction, universal design of learning, opportunities for students, teaching accessibility, and more will be shared.

With the growing awareness of emerging technologies' impacts on teens' lives, families, and communities, rethinking the ways in which we educate and talk about these innovations and their moral and ethical complexities are gaining steam. We present a novel pedagogical intervention that blends techniques from Philosophy for Children (P4C), a pedagogical approach that teaches youth reasoning and argumentative skills, with Youth as Philosophers of Technology, a framework for computing education that foregrounds learning how to decode and unmake tech's relationship with power through artistic, moral and humanistic inquiry, without devaluing core computing practices, such as design, making, coding, and tinkering. We studied this intervention in a summer elective class with 12 students ages 14--18 in the US. Our ongoing data analysis revealed two categories of themes: (1) 'launchpads for ethical sensemaking', namely instances when we observed ethical sensemaking around technology, and (2) 'expressions of ethical sensemaking', namely what students' ethical sensemaking looked like when discussing the ethical implications of technology. We hope to catalyze discussions for both researchers on characterizations of and growth around ethical sensemaking of technology, as well as practitioners on implementations of Youth as Philosophers of Technology and P4C ideas in their classrooms.

This Birds-of-a-Feather (BOF) session will offer LGBTQ+ researchers and practitioners at SIGCSE TS a space for networking, sharing experiences, and being in community together. Given the consistent marginalization of the LGBTQ+ community, this BOF aims to create a community at SIGCSE of professors, students, researchers, and practitioners with interest in computing education. The BOF will provide a space for people to decompress from the long conference day through playing board games and socializing. There is also space for attendees to discuss ways computing education and SIGCSE could be more inclusive to the LGBTQ+ community.

Prior research has investigated children’s perceptions of algorithmic bias, but provides little guidance on engaging children in conversations on algorithmic bias that center their agency and well-being. To address this, we developed discussions and design activities based on three scenarios of algorithmic (un)fairness. We conducted these discussions and activities with 16 children (ages 8-12) in the US, and examined our data using qualitative thematic analysis. Grounded in lived experiences and situated knowledge, participants were capable of reasoning around both explicit and implicit effects of algorithmic bias. Participants also expressed distrust of technology, doubting technology’s abilities and preferring human approaches to resolve unfairness. This work contributes (1) a more nuanced understanding of children’s situated reasoning of technology, suggesting their potential for critical engagement and (2) a blueprint for engaging children in scaffolded yet open-ended sensemaking around algorithmic fairness, informing the design of tools, curricula, and other learning experiences for children.

Automated hiring algorithms are increasingly used in computing job recruitment. Prior work has examined perceptions of algorithmic fairness and established bias in hiring algorithms, but there is limited work on the ability of computer science students, who are applying for their first computing job, to overcome new barriers posed by automated hiring. To investigate what challenges students face, how they work through them, and their perceptions of these systems, we conducted semi-structured interviews with post-secondary students who were first-time computing job applicants. Analyses revealed that participants had diverse knowledge of hiring algorithms; some people knew to use strategies, such as keywords in resumes, online assessment practice, and referrals to circumvent automated processes to progress to in-person interviews, but others were entirely unaware of the automation. Participants also expressed that current systems prevented them from demonstrating the full extent of their skills and attributed job offers to personal contacts within the company. While some deemed automation a "necessary evil" to combat scale, many struggled with the inequity automated hiring processes perpetuated. Understanding student experiences and perspectives with automated hiring has relevance for how current computer science curricula prepares students for the transition to computing jobs post-graduation. Our findings have implications for how to develop new practices to better support students in their transitions amid a changing hiring landscape.

Motivation. All students studying Computer Science (CS) deserve to feel a sense of belonging. In a post-secondary CS class, undergraduate Teaching Assistants (TAs) have the majority of student contact hours, making student-TA interactions, such as those during office hours, important in shaping student belonging. Therefore, we sought to understand student and TA conceptions of belonging, their narratives about their journeys of belonging in CS, and how TAs influence student sense of belonging through office hour interactions.

With the ubiquity of computing technologies, adolescents are increasingly affected by algorithmic biases. While previous work provides insight into adolescents’ perceptions of algorithmic bias, few provide guidance on how to engage adolescents in discourse on algorithmic bias that prioritizes both their agency and safety. To address this, we developed and conducted group discussions and design activities based on three scenarios of algorithmic bias with 15 adolescents of color (ages 15-17) in a summer academic program in the United States targeted at students from families with low-income backgrounds or who would be the first in their family to pursue post-secondary education. When sensemaking, all participants considered factors beyond the scenarios, using their situated knowledge to contextualize perceptions of unfairness. They also considered sources of bias and impacts of unfairness at different levels of individuals, communities, and society. However, when designing solutions, they tended to design for hypothetical “average users” instead of considering nuances of user populations. We offer insights for algorithmic fairness learning experiences that support situated reasoning in adolescents.

Learning programming benefits from self-regulation, but novices lack support for developing these skills of cognitive control. To support their development, we designed Code Replayer, an online tool that enables novice programmers to practice programming and then replay their coding process to reflect and identify process improvements. To evaluate the impact of replaying code on self-regulation, we conducted a formative qualitative evaluation with 21 novice programmers who used Code Replayer to practice writing code. We found that after watching code replays, participants more frequently interpreted problem prompts and planned their solutions, two crucial self-regulation behaviors that novices often overlook. We interpret our results by focusing on two focal points in the design of code replays as a programming self-regulation intervention: interpreting pauses in replays and ensuring replays of struggle are more informative and less detrimental.

Decision-making is a key software engineering skill. Developers constantly make choices throughout the software development process, from requirements to implementation. While prior work has studied developer decision-making, the choices made while choosing what solution to write in code remain understudied. In this mixed-methods study, we examine the phenomenon where developers select one specific way to implement a behavior in code, given many potential alternatives. We call these decisions implementation design decisions. Our mixed-methods study includes 46 survey responses and 14 semi-structured interviews with professional developers about their decision types, considerations, processes, and expertise for implementation design decisions. We find that implementation design decisions, rather than being a natural outcome from higher levels of design, require constant monitoring of higher level design choices, such as requirements and architecture. We also show that developers have a consistent general structure to their implementation decision-making process, but no single process is exactly the same. We discuss the implications of our findings on research, education, and practice, including insights on teaching developers how to make implementation design decisions.

Teachers are essential to equitably broadening participation in computing in schools, but the creation of CS teacher education pathways faces many challenges. In this experience report, we share the many political, administrative, institutional, and sustainability barriers our institution faced in creating a secondary CS pre-service pathway. Throughout, we discuss the particular design choices we made in order to center equity and justice, both in the content of the program, but also in its structure, policies, and resources, which were often in tension with state standards and policies. We also describe our experience teaching and supporting the inaugural cohort of graduates as well as the graduates' experiences, which revealed tension between utopian and dystopian futures of computing and their role in helping students navigate them. We end with a reflection on key factors that we believe led to its successful first year launch, including leadership, interdisciplinarity, capacity, timing, and funding, and on sustainability concerns, including tuition subsidy and instructional capacity.

Students with disabilities face a variety of challenges in computer science education including those related to stigma around disability, inaccessible tools and instruction, disability disclosure, and a lack of mentors. This BOF will bring together individuals who are interested in increasing the representation of students with disabilities in computing and improving their success. Participants will share strategies to help each other do a better job of including these students in our classes and research projects. Resources related to accessible tools and instruction, universal design of learning, opportunities for students, and more will be shared.

Technology should be accessible and inclusive, so designers should learn to consider the needs of different users. Toward this end, we created the theoretically-grounded CIDER assumption elicitation technique, an educational analytical design evaluation method to teach inclusive design skills. CIDER ( Critique , Imagine , Design , Expand , Repeat ) helps designers recognize and respond to bias using the critical lens of assumptions about users . Through an 11-week mixed-method case study in an interaction design course with 40 undergraduate students and follow-up interviews, we found that activities based on the CIDER technique may have helped students identify increasingly many types of design bias over time and reflect on their unconscious biases about users. The activities also had lasting impacts, encouraging some students to adopt more inclusive approaches in subsequent design work. We discuss the implications of these findings, namely that educational techniques like CIDER can help designers learn to create equitable technology designs.

Prior work has broadly explored empowering children to learn to program by making video games. However, such work has rarely considered the role of families in this learning, leaving many open questions about how inter-generational collaborations might support and constrain learning. To investigate these opportunities, we conducted a family-based study of TileCode, a rule-based programming platform for video-game programming, and scaffolded a 4-week series of game programming activities with 19 children (9 to 14 years old) and 16 parents. Using a joint media engagement lens to analyze family knowledge and programming strategies, we found: 1) families demonstrated many dynamic collaboration patterns distinct from pair programming and other collaboration models, 2) parents played a unique role in scaffolding and guiding more complex designs and programming tasks, 3) families found it challenging to start their games from scratch but benefited greatly from having programming patterns for particular game behaviors. These findings suggest the need for game programming platforms to design around the unique kinds of collaboration in inter-generational domain-specific programming.

In many domains, strategic knowledge is documented and shared through checklists and handbooks. In software engineering, however, developers rarely share strategic knowledge for approaching programming problems, in contrast to other artifacts and despite its importance to productivity and success. To understand barriers to sharing, we simulated a programming strategy knowledge-sharing platform, asking experienced developers to articulate a programming strategy and others to use these strategies while providing feedback. Throughout, we asked strategy authors and users to reflect on the challenges they faced. Our analysis revealed that developers could share strategic knowledge. However, they struggled in choosing a level of detail and understanding the diversity of the potential audience. While authors required substantial feedback, users struggled to give it and authors to interpret it. Our results suggest that sharing strategic knowledge differs from sharing code and raises challenging questions about how knowledge-sharing platforms should support search and feedback.

Many families engage daily with artificial intelligence (AI) applications, from conversations with a voice assistant to mobile navigation searches. While there are known ways for youth to learn about AI, we do not yet understand how to engage parents in this process. To explore parents’ roles in helping their children develop AI literacies, we designed 11 learning activities organized into four topics: image classification, object recognition, interaction with voice assistants, and unplugged AI co-design. We conducted a 5-week online in-home study with 18 children (5 to 11 years old) and 16 parents. We identify parents’ most common roles in supporting their children and consider the benefits of parent-child partnerships when learning AI literacies. Finally, we discuss how our different activities supported parents’ roles and present design recommendations for future family-centered AI literacies resources.

Objectives. Teachers are essential to making computing education available to students. A key place to sustainably prepare computer science (CS) teachers is in pre-service preparation programs, which are often required for certification in the United States. Prior work has examined many reasons that people choose to become teachers — or choose not to — but little prior work has examined factors that shape the pursuit of CS certifications in pre-service in particular.

Computing education research (CER) has used demographic data to understand learners’ identities, backgrounds, and contexts for efforts such as culturally-responsive computing. Prior work indicates that failing to elucidate and critically engage with the implicit assumptions of a field can unintentionally reinforce power structures that further marginalize people from non-dominant groups. The goal of this paper is two-fold: to understand what populations CER researchers have studied, and to surface implicit assumptions about how researchers have collected, reported, and used demographic data on these populations. We conducted a content analysis of 510 peer-reviewed papers published in 12 CER venues from 2012 to 2021. We found that (1) 60% of papers studied older learners in formal contexts (i.e. post-secondary education); (2) 68% of papers left unclear how researchers collected demographic data; and (3) while 94% of papers were single-site studies, only 14% addressed the limitations of their contexts. We also identified hegemonic norms through ambiguous aggregate term usage (e.g. underrepresented, diverse) in 23% of papers, and through incomplete reporting of demographics (i.e. leaving out demographics for some participants in their sample) in 35% of papers. We discuss the implications of these findings for the CER field, raising considerations for CER researchers to keep in mind when collecting, reporting, and using demographic data.

Artificial Intelligence (AI) educational resources such as training tools, interactive demos, and dedicated curriculum are increasingly popular among educators and learners. While prior work has examined pedagogies for promoting AI literacy, it has yet to examine how well technology resources support these pedagogies. To address this gap, we conducted a systematic analysis of existing online resources for AI education, investigating what learning and teaching affordances these resources have to support AI education. We used the Technological Pedagogical Content Knowledge (TPACK) framework to analyze a final corpus of 50 AI resources. We found that most resources support active learning, have digital or physical dependencies, do not include all the five big ideas defined by AI4K12 guidelines, and do not offer built-in support for assessment or feedback. Teaching guides are hard to find or require technical knowledge. Based on our findings, we propose that future AI curricula move from singular activities and demos to more holistic designs that include support, guidance, and flexibility for how AI technology, concepts, and pedagogy play out in the classroom.

In order to advance broadening participation in computing (BPC) efforts, research and practice projects should implement a board (advisory or strategic) that specifically focuses on equity. In light of the murder of George Floyd, the impact of COVID-19 on minoritized communities, and the historical (and continual) erasure of diverse voices within the computing and technology fields, it is important that networks, alliances, and the broader computer science community engage with stakeholders with critical expertise in areas of diversity, equity, and inclusion. In this panel, Expanding Computing in Education Pathways (ECEP) Alliance Executive Board members will discuss the need for equity-centered conversations, specifically when it comes to BPC. Panelists and audience members will engage in dialogue around the necessity for, and value of, creating spaces where diverse voices are elevated and supported.

The session will present an overview of findings of a recently funded NSF workshop that set out to examine the pressing issues for computing education research for the next 15 years. Based on dialogs for scholars working in this area, the workshop participants developed a series of themes and topics that they felt should become the focus of computing education research efforts for the next 15 years. Main themes that emerged and will be discussed in this session include: diversity, equity, inclusion, ethics, broadening participation, teaching, learning, K-12, research to practice, computing's connection to other fields, and computing education research disciplinary issues. The session will focus on interesting research questions from each of these areas that are ripe to be explored as well as enablers and blockers to the progress of this work.

Part of broadening participation in computer science (CS) is understanding what experiences and identities students bring with them to the classroom and building upon them. Prior work has often achieved this by connecting CS concepts to cultural ideas and practices. Increasingly, however, youth may be encountering sociotechnical and sociopolitical counternarratives about computing, power, and justice, offering new opportunities to connect CS to students' lives. To understand what role these emerging counternarratives have in secondary CS classrooms, we taught a co-constructed high school course to a racially, ethnically, socioeconomically, and gender diverse classroom, framing the course as both a creative and critical introduction to CS, giving agency to students to incorporate critical themes into their learning. We gathered notes, artifacts, and student responses over the course of 6-weeks, and analyzed the extent to which students brought critical themes into their creative work, developing critical consciousness of CS concepts. We found that before there was space for critical conversations about computing, we had to navigate students' issues of trust, positionality, and the broader inequitable systems of education in which the class occurred. Only after navigating those tensions did students feel safe to have those critical conversations. Once they did, they rapidly embraced the counternarratives, structured their learning around them, and used them to build community and support each other.

Social upheaval through widespread disinformation, aggressive automation, and algorithmic oppression have led to an increasing focus on the ethical considerations of technologists. In response, researchers and educators have looked to integrate ethics into Computer Science curricula, either by creating ethics-exclusive courses or embedding ethics into existing computing topics. Regardless of approach, few ethics integrations seek to explicitly center counternarratives, narratives opposing dominant narratives within computing, as a method of instruction. Given an existing teaching opportunity, our prior experience with computer systems education, and a lack of existing ethics integrations into computer systems, we integrated counternarratives into an introductory systems course. We framed this integration through theHouse of Computing (HoC), a structural metaphor that frames the computing discipline as an object for critique. Throughout the course, we presented counternarratives alongside technical content. We assessed student understanding of counternarratives through "floorplans'': metaphorical representations of course units, or floors within theHoC. Through an analysis of students' first floorplan, we found that nearly every student expressed existing or newfound awareness of structural problems within computing, though the novelty of the floorplans concept challenged students. Based on this experience, we offer recommendations for instructors looking to teach computer systems critically or integrate counternarratives into other computing courses.

Students with disabilities face a variety of challenges including those related to stigma around disability, inaccessible tools and instruction, disability disclosure, and a lack of mentors. This BOF will bring together individuals who are interested in increasing the representation of students with disabilities in computing and improving their success. Participants will share strategies to help each other do a better job of including these students in our classes and research projects. Resources related to accessible tools and instruction, universal design of learning, opportunities for students, and more will be shared.

The pressing need to produce more computer science graduates in the United States to meet the ever-growing number of open computing and technology jobs, and the lack of parity in gender representation necessitates identifying measures to help retain and support more seekers of computer science majors. Fostering a strong sense of belonging to the field of computing has been shown to increase persistence in the major, particularly for women and people of color. In this panel, we will discuss techniques for building community along with personal experiences regarding the impact of having and also lacking that community. Current research on belonging and its impact on persistence will also be presented.

As computing courses become larger, students of minoritized groups continue to disproportionately face challenges that hinder their academic and professional success (e.g. implicit bias, microaggressions, lack of resources, assumptions of preparatory privilege). This can impact career aspirations and sense of belonging in computing communities. Instructors have the power to make immediate changes to support more equitable learning, but they are often unaware of students' challenges. To help both instructors and students understand the inequities in their classes, we developed StudentAmp, an interactive system that uses student feedback and self-reported demographic information (e.g. gender, ethnicity, disability, educational background) to show challenges and how they affect students differently. To help instructors make sense of feedback, StudentAmp ranks challenges by student-perceived disruptiveness. We conducted formative evaluations with five large college computing courses (150 - 750 students) being taught remotely during the COVID-19 pandemic. We found that students shared challenges beyond the scope of the course, perceived sharing information about who they were as useful but potentially dangerous, and that teaching teams were able to use this information to consider the positionality of students sharing challenges. Our findings relate to a central design tension of supporting equity by sharing contextualized information about students while also ensuring their privacy and well-being.

Children are increasingly surrounded by AI technologies but can overestimate smart devices’ abilities due to their lack of transparency. Drawing on the sense-making theory, this study explores how children come to see machine intelligence after training custom machine learning models and creating smart programs that use them. Through a 4-week observational study in after-school programs with 52 children (7 to 12 years old), we found that children engage in the scientific method while training, coding and testing their smart programs. We also found that children became more skeptical of certain abilities of smart devices as they shifted their attribution of agency from the devices to the people who program them. These changes in perception happened both through individual interactions with agents and prompted debates with peers. Based on these results, we conclude with discussions on strategies for promoting children’s sense-making practices and sense of agency in the age of machine learning.

Modern visualization tools aim to allow data analysts to easily create exploratory visualizations. When the input data layout conforms to the visualization design, users can easily specify visualizations by mapping data columns to visual channels of the design. However, when there is a mismatch between data layout and the design, users need to spend significant effort on data transformation.

Understanding inequity at scale is necessary for designing equitable online learning experiences, but also difficult. Statistical techniques like differential item functioning (DIF) can help identify whether items/questions in an assessment exhibit potential bias by disadvantaging certain groups (e.g. whether item disadvantages woman vs man of equivalent knowledge). While testing companies typically use DIF to identify items to remove, we explored how domain-experts such as curriculum designers could use DIF to better understand how to design instructional materials to better serve students from diverse groups. Using Code.org's online Computer Science Discoveries (CSD) curriculum, we analyzed 139,097 responses from 19,617 students to identify DIF by gender and race in assessment items (e.g. multiple choice questions). Of the 17 items, we identified six that disadvantaged students who reported as female when compared to students who reported as non-binary or male. We also identified that most (13) items disadvantaged AHNP (African/Black, Hispanic/Latinx, Native American/Alaskan Native, Pacific Islander) students compared to WA (white, Asian) students. We then conducted a workshop and interviews with seven curriculum designers and found that they interpreted item bias relative to an intersection of item features and student identity, the broader curriculum, and differing uses for assessments. We interpreted these findings in the broader context of using data on assessment bias to inform domain-experts' efforts to design more equitable learning experiences.

Reliable and valid exams are a crucial part of both sound research design and trustworthy assessment of student knowledge. Assessing and addressing item bias is a crucial step in building a validity argument for any assessment instrument. Despite calls for valid assessment tools in CS, item bias is rarely investigated. What kinds of item bias might appear in conventional CS1 exams? To investigate this, we examined responses to a final exam in a large CS1 course. We used differential item functioning (DIF) methods and specifically investigated bias related to binary gender and year of study. Although not a published assessment instrument, the exam had a similar format to many exams in higher education and research: students are asked to trace code and write programs, using paper and pencil. One item with significant DIF was detected on the exam, though the magnitude was negligible. This case study shows how to detect DIF items so that future researchers and practitioners can do these analyses.

Students with disabilities face a variety of challenges including those related to stigma around disability, inaccessible tools and instruction, disability disclosure, and a lack of mentors. This BOF will bring together individuals who are interested in increasing the representation of students with disabilities in computing and improving their success. Participants will share strategies to help each other do a better job of including these students in our classes and research projects. Resources related to accessible tools and instruction, universal design of learning, opportunities for students, and more will be shared.

Developers rely heavily on resources to find technical insights on how to use languages, APIs, and platforms, seeking help from Stack Overflow, GitHub, meetups, blogs, live streams, forums, documentation, and more. However, there is one kind of knowledge for which resources are hard to find: strategic knowledge. In contrast to technical knowledge, strategic knowledge provides insight into how to approach problem-solving. Prior work has demonstrated that developers can make use of written strategies to improve their problem-solving. However, there is currently no way for developers to share, curate, and search for this knowledge at scale. To address this gap, we contribute HowToo, a platform for sharing, finding, and using programming strategies. Its key insight is that there are many different approaches to the same problem, and developers may need different strategies depending on their situation. In a longitudinal evaluation with more than 30 students in a project-based software engineering course, we found that: 1) students viewed HowToo as complementary to technical resources; 2) students viewed strategies as helping them be more systematic and complete in their work; 3) HowToo helped students be more confident in their problem solving; 4) when students were under time pressure, they were less inclined to use HowToo to structure their work, as being mindful required them to slow down.

Design is a distinct discipline with its own practices, tools, professions, and areas of scholarship. However, practitioners from other fields often leverage aspects of design in their own work, leading to subfields like engineering design and architecture design that are neither wholly design nor wholly the intersecting discipline. Similarly, design and computing are known to intersect in educational contexts. Unfortunately, we do not yet have a clear understanding of how to characterize the kinds of design that may accompany computing topics, resulting in challenges to teaching and learning. This gap is particularly prevalent in K-12 computing education, where design is often used to promote student engagement but rarely studied as its own disciplinary phenomenon. Toward the goal of better understanding the nature and role of design in computing education, this article motivates and describes two qualitative, exploratory analyses of how design skills manifest in popular K-12 computing education curricula and activities. We find evidence to suggest two types of design within existing computing education curricula and standards: nondisciplinary problem-space design , which deals with defining software requirements, and disciplinary program-space design , which deals with choosing how best to meet those requirements. We find that these two types of computing design may exist independently, but they often overlap, creating an intriguing intersection of discipline-specific computing design educational activity. Finally, we discuss the practical implications of proceeding with research and educational practice in light of these results, highlighting the need for further exploration into the unique overlap of design and computing education.

Creating modern software inevitably requires using application programming interfaces (APIs). While software developers can sometimes use APIs by simply copying and pasting code examples, a lack of robust knowledge of how an API works can lead to defects, complicate software maintenance, and limit what someone can express with an API. Prior work has uncovered the many ways that API documentation fails to be helpful, though rarely describes precisely why. We present a theory of robust API knowledge that attempts to explain why, arguing that effective understanding and use of APIs depends on three components of knowledge: (1) the domain concepts the API models along with terminology, (2) the usage patterns of APIs along with rationale, and (3) facts about an API’s execution to support reasoning about its runtime behavior. We derive five hypotheses from this theory and present a study to test them. Our study investigated the effect of having access to these components of knowledge, finding that while learners requested these three components of knowledge when they were not available, whether the knowledge helped the learner use or understand the API depended on the tasks and likely the relevance and quality of the specific information provided. The theory and our evidence in support of its claims have implications for what content API documentation, tutorials, and instruction should contain and the importance of giving the right information at the right time, as well as what information API tools should compute, and even how APIs should be designed. Future work is necessary to both further test and refine the theory, as well as exploit its ideas for better instructional design.

Software developers often make interface design decisions and work with designers. Therefore, computing students who seek to become developers need some education about interface design. While prior work has studied difficulties that educators face when teaching design to computing students, there is comparatively little work on the difficulties computing students face when learning HCI design skills. To uncover these difficulties, we conducted two qualitative studies consisting of surveys and interviews with (1) computing students and (2) educators who teach interface design to computing students. Qualitative analysis of their responses revealed 18 types of learning difficulties students might experience in HCI design education, including difficulties around the mechanics of design work, project management skills, the wicked nature of design problems, and distorted perspectives on design.

Although exploring alternatives is fundamental to creating better interface designs, current processes for creating alternatives are generally manual, limiting the alternatives a designer can explore. We present Scout, a system that helps designers rapidly explore alternatives through mixed-initiative interaction with high-level constraints and design feedback. Prior constraint-based layout systems use low-level spatial constraints and generally produce a single design. Tosupport designer exploration of alternatives, Scout introduces high-level constraints based on design concepts (e.g.,~semantic structure, emphasis, order) and formalizes them into low-level spatial constraints that a solver uses to generate potential layouts. In an evaluation with 18 interface designers, we found that Scout: (1) helps designers create more spatially diverse layouts with similar quality to those created with a baseline tool and (2) can help designers avoid a linear design process and quickly ideate layouts they do not believe they would have thought of on their own.

Machine learning systems are increasingly a part of everyday life, and often used to make critical and possibly harmful decisions that affect stakeholders of the models. Those affected need enough literacy to advocate for themselves when models make mistakes. To understand how to develop this literacy, this paper investigates three ways to teach ML concepts, using linear regression and gradient descent as an introduction to ML foundations. Those three ways include a basic Facts condition, mirroring a presentation or brochure about ML, an Impersonal condition which teaches ML using some hypothetical individual's data, and a Personal condition which teaches ML on the learner's own data in context. Next, we evaluated the effects on learners' ability to self-advocate against harmful ML models. Learners wrote hypothetical letters against poorly performing ML systems that may affect them in real-world scenarios. This study discovered that having learners learn about ML foundations with their own personal data resulted in learners better grounding their self-advocacy arguments in the mechanisms of machine learning when critiquing models in the world.

Choices learners make when navigating a self-directed online learning tool can impact the effectiveness of the experience. But these tools often do not afford learners the agency or the information to make decisions beneficial to their learning. We evaluated the effect of varying levels of information and agency in a self-directed environment designed to teach programming. We investigated three design alternatives: informed high-agency, informed low-agency, and less informed high-agency. To investigate the effect of these alternatives on learning, we conducted a study with 79 novice programmers. Our results indicated that increased agency and information may have translated to more motivation, but not improved learning. Qualitative results suggest this was due to the burden that agency and information placed on decision-making. We interpret our results in relation to informing the design of self-directed online tools for learner agency.

Opportunities for training CS K-12 pre-service and in-service teachers, research in CS Education, and career pathways for PhDs/EdDs in CS education are happening, but often in an uncoordinated way. We advocate that now is the right time for CS and Education to collaborate on developing new joint degree programs in Computer Science Education and to explore joint faculty appointments. High undergraduate enrollment in computing programs and the increasing interest in CS courses from non-majors represent a unique opportunity for starting successful programs. As more of CS undergraduates are undergraduate TAs and see teaching and learning from a non-learner perspective, their interest in education has also increased. The growing interest in CS education, including the need for effecting CS teaching at both K-12 and the undergraduate level, provide interesting job opportunities for CS education researchers. As CS departments develop new undergraduate degree programs and scale class sizes, research on questions like How do we teach effectively computing to different audiences? How can we assess CS learning? What are culturally responsive pedagogies? is important. To answer many of these and related questions, CS departments should be actively engaged in CS Education research, from training graduate students in interdisciplinary programs to research programs. This BOF will provide a platform for the discussion on what such graduate programs - from certificate to a PhD - can and should look like, what challenges exist to creating them, and how students with different backgrounds should get trained in the relevant foundations of CS and Education.

This panel will provide practical advice on what and how to teach accessibility in a variety of settings. In this context, teaching accessibility means teaching about computer technologies that people with various disabilities can use and be productive with. At the undergraduate level it could mean teaching about how to design and build accessible web sites and applications. At the graduate level it could be teaching about building applications that can help people with disabilities with specific tasks. An entire course could focus on accessibility or it could be just part of an existing course. It is also important to learn about the diversity of consumers of technologies: what their abilities are and what access infrastructures they use every day. All the panelists have extensive experience in teaching accessibility. They will provide the audience of the panel deep insights into what they might do to teach accessibility in their own courses.

Approximately 10% of computer science and engineering majors have a disability. Students with disabilities face a variety of challenges including those related to stigma around disability, inaccessible tools and instruction, disability disclosure, and a lack of mentors. This BOF will bring together individuals who are interested in increasing the representation of students with disabilities in computing and improving their success. Participants will share strategies to help each other do a better job of including these students in our classes and research projects. Resources related to accessible tools and instruction, universal design of learning, opportunities for students, and more will be shared.

The 32 chapters of the 2019 Cambridge Handbook of Computing Education Research synthesize the existing research in computing education and propose new directions for future research. An author from each chapter will summarize their chapter with auto-advancing slides. Attendees will be introduced to the breadth of content in the new handbook and can identify chapters of interest. This fits uniquely as a special session, and will likely be informative, inspiring, and overwhelming.

Prior work on novice programmers' self-regulation have shown it to be inconsistent and shallow, but trainable through direct instruction. However, prior work has primarily studied self-regulation retrospectively, which relies on students to remember how they regulated their process, or in laboratory settings, limiting the ecological validity of findings. To address these limitations, we investigated 31 novice programmers' self-regulation in situ over 10 weeks. We had them to keep journals about their work and later had them to reflect on their journaling. Through a series of qualitative analyses of journals and survey responses, we found that all participants monitored their process and evaluated their work, that few interpreted the problems they were solving or adapted prior solutions. We also found that some students self-regulated their programming in many ways, while others in almost none. Students reported many difficulties integrating reflection into their work; some were completely unaware of their process, some struggled to integrate reflection into their process, and others found reflection conflicted with their work. These results suggest that self-regulation during programming is highly variable in practice, and that teaching self-regulation skills to improve programming outcomes may require differentiated instruction based on students self-awareness and existing programming practices.

Most CS students learn little about accessibility in higher education; this is partly because most CS faculty know little about accessibility. Unfortunately, higher education CS faculty lack a model of professional development for learning to teach new topics. Therefore, we investigated the feasibility of a "micro" professional development model for teaching accessibility in CS courses that could be used at scale. We conducted 18 semi-structured interviews with U.S. CS faculty, asking them to explore a prototype of a web-based professional development tool that linked accessibility topics to CS topics. We found that many organizational factors limited faculty's autonomy to integrate accessibility in many of their courses. We also found that individual values and knowledge constrained faculty's ability and willingness to both learn and integrate accessibility topics into their courses. However, many faculty expressed desire to teach accessibility in their courses if they had access to even basic accessibility content and materials to use in their courses.

One way to teach programming problem solving is to teach explicit, step-by-step strategies. While prior work has shown these to be effective in controlled settings, there has been little work investigating their efficacy in classrooms. We conducted a 5-week case study with 17 students aged 15-18, investigating students' sentiments toward two strategies for debugging and code reuse, students' use of scaffolding to execute these strategies, and associations between students' strategy use and their success at independently writing programs in class. We found that while students reported the strategies to be valuable, many had trouble regulating their choice of strategies, defaulting to ineffective trial and error, even when they knew systematic strategies would be more effective. Students that embraced the debugging strategy completed more features in a game development project, but this association was mediated by other factors, such as reliance on help, strategy self-efficacy, and mastery of the programming language used in the class. These results suggest that teaching of strategies may require more explicit instruction on strategy selection and self-regulation.

Approximately 10% of computer science and engineering majors have a disability. Students with disabilities are more likely to drop out of the major than those without disabilities. At the K-12 level, many tools used to teach computing have limited accessibility to students with disabilities. This BOF will bring together individuals who are interested in increasing the representation of students with disabilities in computing and improving their success. Participants will share strategies to help each other do a better job of including these students in our classes and research projects. Resources, including those produced by AccessComputing and AccessCSforAll, will be shared.

Tests serve an important role in computing education, measuring achievement and differentiating between learners with varying knowledge. But tests may have flaws that confuse learners or may be too difficult or easy, making test scores less valid and reliable. We analyzed the Second Computer Science 1 (SCS1) concept inventory, a widely used assessment of introductory computer science (CS1) knowledge, for such flaws. The prior validation study of the SCS1 used Classical Test Theory and was unable to determine whether differences in scores were a result of question properties or learner knowledge. We extended this validation by modeling question difficulty and learner knowledge separately with Item Response Theory (IRT) and performing expert review on problematic questions. We found that three questions measured knowledge that was unrelated to the rest of the SCS1, and four questions were too difficult for our sample of 489 undergrads from two universities.

As the societal demands for application and knowledge in computer science (CS) increase, CS student enrollment keeps growing rapidly around the world. By continuously improving the efficacy of computing education and providing guidelines for learning and teaching practice, computing education research plays a vital role in addressing both educational and societal challenges that emerge from the growth of CS students. Given the significant role of computing education research, it is important to ensure the reliability of studies in this field. The extent to which studies can be replicated in a field is one of the most important standards for reliability. Different fields have paid increasing attention to the replication rates of their studies, but the replication rate of computing education was never systematically studied. To fill this gap, this study investigated the replication rate of computing education between 2009 and 2018. We examined 2,269 published studies from three major conferences and two major journals in computing education, and found that the overall replication rate of computing education was 2.38%. This study demonstrated the need for more replication studies in computing education and discussed how to encourage replication studies through research initiatives and policy making.

Research suggests that parental engagement through Joint Media Engagement (JME) is an important factor in children's learning for coding and programming. Unfortunately, parents with limited technology background may have difficulty supporting their children's access to programming. English-language learning (ELL) families from marginalized communities face particular challenges in understanding and supporting programming, as code is primarily authored using English text. We present BlockStudio, a programming tool for empowering ELL families to jointly engage in introductory coding, using an environment embodying two design principles, text-free and visually concrete. We share a case study involving three community centers serving immigrant and refugee populations. Our findings show ELL families can jointly engage in programming without text, via co-creation and flexible roles, and can create a range of artifacts, indicating understanding of aspects of programming within this environment. We conclude with implications for coding together in ELL families and design ideas for text-free programming research.

Interface designers often use screenshot images of example designs as building blocks for new designs. Since images are unstructured and hard to edit, designers typically reconstruct screenshots with vector graphics tools in order to reuse or edit parts of the design. Unfortunately, this reconstruction process is tedious and slow. We present Rewire, an interactive system that helps designers leverage example screenshots. Rewire automatically infers a vector representation of screenshots where each UI component is a separate object with editable shape and style properties. Based on this representation, the system provides three design assistance modes that help designers reuse or redraw components of the example design. The results from our quantitative and user evaluations demonstrate that Rewire can generate accurate vector representations of interface screenshots found in the wild and that design assistance enables users to reconstruct and edit example designs more efficiently compared to a baseline design tool.

About half of recent computer and information science graduates attended community college at some point. Prior work on transfer students in general suggests that the transfer process can engage people from underrepresented communities, but can also be academically and socially "shocking". However, we know little about the experiences of transfer students in computer science in particular. We used the Laanan-Transfer Student Questionnaire (L-TSQ) to survey 25 transfer students and 135 native (non-transfer) students and conducted follow-up interviews with 8 transfer students attending a large public 4-year university in a city with significant technology industry presence. We found that while transfer students were more diverse demographically, the support of the university for transfer student orientation tended to mitigate social shocks of transferring. This did not, however, eliminate gaps in academic performance. These findings suggest that there are other non-social factors that influence academic performance that CS programs must support to equitably engage students who transfer.

A primary goal of computing education research is to discover designs that produce better learning of computing. In this pursuit, we have increasingly drawn upon theories from learning science and education research, recognizing the potential benefits of optimizing our search for better designs by leveraging the predictions of general theories of learning. In this paper, we contribute an argument that theory can also inhibit our community's search for better designs. We present three inhibitions: 1) our desire to both advance explanatory theory and advance design splits our attention, which prevents us from excelling at both; 2) our emphasis on applying and refining general theories of learning is done at the expense of domain-specific theories of computer science knowledge, and 3) our use of theory as a critical lens in peer review prevents the publication of designs that may accelerate design progress. We present several recommendations for how to improve our use of theory, viewing it as just one of many sources of design insight in pursuit of improving learning of computing.

Inclusive design is important in today's software industry, but there is little research about how to teach it. In collaboration with 9 teacher-researchers across 8 U.S. universities and more than 400 computer and information science students, we embarked upon an Action Research investigation to gather insights into the pedagogical content knowledge (PCK) that teachers need to teach a particular inclusive design method called GenderMag. Analysis of the teachers' observations and experiences, the materials they used, direct observations of students' behaviors, and multiple data on the students' own reflections on their learning revealed 11 components of inclusive design PCK. These include strategies for anticipating and addressing resistance to the topic of inclusion, strategies for modeling and scaffolding perspective taking, and strategies for tailoring instruction to students' prior beliefs and biases.

The program comprehension and mining software repository communities are, in practice, two separate research endeavors. One is concerned with what's happening in a developer's mind, while the other is concerned with what's happening in a team. And yet, implicit in these fields is a common goal to make better software and the common approach of influencing developer decisions. In this keynote, I provide several examples of this overlap, suggesting several grand challenges in comprehension and mining.

The program comprehension and mining software repository communities are, in practice, two separate research endeavors. One is concerned with what's happening in a developer's mind, while the other is concerned with what's happening in a team. And yet, implicit in these fields is a common goal to make better software and the common approach of influencing developer decisions. In this keynote, I provide several examples of this overlap, suggesting several grand challenges in comprehension and mining.

Influencing adolescent interest in computing is key to engaging diverse teens in computer science learning. Prior work suggests that informal mentorship may be a powerful way to trigger and maintain interest in computing, but we still know little about how mentoring relationships form, how mentors trigger and maintain interest, or what qualities adolescents value in informal mentors. In a 3-week career exploration class with 18 teens from underrepresented groups, we had students write extensively about their informal computing mentors. In analyzing their writing, we found that most teens had informal computing mentors, that mentors were typically teachers, friends, and older siblings (and not parents or school counselors), and that what teens desired most were informal mentors that were patient, helpful, inspiring, and knowledgeable. These findings suggest that computing mentors can come in many forms, that they must be patient, helpful, and inspiring, but that they also require content knowledge about computing to be meaningful. Future work might explore what knowledge of computing is sufficient to empower teachers, parents, peers, and family to be effective computing mentors.

Industry demand for software developers with knowledge of accessibility has increased substantially in recent years. However, there is little knowledge about the prevalence of higher education teaching about accessibility or faculty's perceived barriers to teaching accessibility. To address this gap, we surveyed 14,176 computing and information science faculty in the United States. We received a representative sample of at least one response from 318 of the 352 institutions we surveyed, totaling 1,857 responses. We found that 175 institutions (50%) had at least one instructor teaching accessibility and that no fewer than 2.5% of faculty overall teach accessibility. Faculty that teach accessibility are twice as likely to be female, to have expertise in HCI and software engineering, and to know people with disabilities. The most critical barriers to teaching accessibility that faculty reported were the absence of clear and discipline-specific accessibility learning objectives and the lack of faculty knowledge about accessibility. Faculty desired resources that were specific to the areas of computing in which they teach rather than general accessibility resources and guidelines.

We propose and evaluate a lightweight strategy for tracing code that can be efficiently taught to novice programmers, building off of recent findings on "sketching" when tracing. This strategy helps novices apply the syntactic and semantic knowledge they are learning by encouraging line-by-line tracing and providing an external representation of memory for them to update. To evaluate the effect of teaching this strategy, we conducted a block-randomized experiment with 24 novices enrolled in a university-level CS1 course. We spent only 5-10 minutes introducing the strategy to the experimental condition. We then asked both conditions to think-aloud as they predicted the output of short programs. Students using this strategy scored on average 15% higher than students in the control group for the tracing problems used the study (p<0.05). Qualitative analysis of think-aloud and interview data showed that tracing systematically (line-by-line and "sketching" intermediate values) led to better performance and that the strategy scaffolded and encouraged systematic tracing. Students who learned the strategy also scored on average 7% higher on the course midterm. These findings suggest that in <1 hour and without computer-based tools, we can improve CS1 students' tracing abilities by explicitly teaching a strategy.

Most web applications are designed as one-size-fits-all, despite considerable variation in people's expertise, physical abilities, and other factors that impact interaction. For example, some web applications require the use of a mouse, precluding use by many people with severe motor disabilities. Other applications require laborious manual input that a skilled developer could automate if the application were scriptable. This paper presents Genie, a system that automatically reverse engineers an abstract model of the underlying commands in a web application, then enables interaction with that functionality through alternative interfaces and other input modalities (e.g., speech, keyboard, or command line input). Genie comprises an abstract model of command properties, behaviors, and dependencies as well as algorithms that reverse engineer this model from an existing web application through static and dynamic program analysis. We evaluate Genie by developing several interfaces that automatically add support for speech, keyboard, and command line input to arbitrary web applications.

Prior work on adolescent interest development shows that mentorship can promote interest in a subject while reshaping beliefs about the subject. To what extent do these same effects occur in computing, where interest and beliefs have traditionally been negative? We conducted two studies of the Puget Sound region in the United States, surveying and teaching 57 diverse adolescents with interests in computing. In the first study, we found that interest in computing was strongly related to having a mentoring relationship and not to gender or socioeconomic status. Teens with mentors also engaged in significantly more computing education and had more diverse beliefs about peers who engaged in computing education. The second study reinforced this finding, showing that teens who took a class from an instructor who aimed to become students' teacher-mentor had significantly greater positive changes in interest in computing than those who already had a mentor. These findings, while correlational, suggest that mentors can play a key role in promoting adolescent interest in computing.

What knowledge does learning programming require? Prior work has focused on theorizing program writing and problem solving skills. We examine program comprehension and propose a formal theory of program tracing knowledge based on control flow paths through an interpreter program's source code. Because novices cannot understand the interpreter's programming language notation, we transform it into causal relationships from code tokens to instructions to machine state changes. To teach this knowledge, we propose a comprehension-first pedagogy based on causal inference, by showing, explaining, and assessing each path by stepping through concrete examples within many example programs. To assess this pedagogy, we built PLTutor, a tutorial system with a fixed curriculum of example programs. We evaluate learning gains among self-selected CS1 students using a block randomized lab study comparing PLTutor with Codecademy, a writing tutorial. In our small study, we find some evidence of improved learning gains on the SCS1, with average learning gains of PLTutor 60% higher than Codecademy (gain of 3.89 vs. 2.42 out of 27 questions). These gains strongly predicted midterms (R2=.64) only for PLTutor participants, whose grades showed less variation and no failures.

Coding bootcamps are a new and understudied way of training new software developers. To learn about the barriers bootcamp students face, we interviewed twenty-six coding bootcamp students and analyzed the interviews using the Communities of Practice framework. We found that bootcamps can be part of an alternate path into the software industry and they provided a second chance for those who missed computing education opportunities earlier, particularly for women. While bootcamps represented a second chance, students entering the industry through bootcamps faced great personal costs and risks, often including significant time, money and effort spent before, during, and after their bootcamps. Though the coursework of bootcamps only ranged from three to six months, career change could take students a year or more, with some students even attending sections of multiple bootcamps.

Online coding tutorials are increasingly popular among learners, but we still have little knowledge of their quality. To address this gap, we derived several dimensions of pedagogical effectiveness from the learning sciences and education literature and analyzed a large sample of tutorials against these dimensions. We sampled 30 popular and diverse online coding tutorials, and analyzed what and how they taught learners. We found that tutorials largely taught similar content, organized content bottom-up, and provided goal-directed practices with immediate feedback. However, few were tailored to learners' prior coding knowledge and only a few informed learners how to transfer and apply learned knowledge. Based on these results, we discuss strengths and weaknesses of online coding tutorials, opportunities for improvement, and recommend that educators point their students to educational games and interactive tutorials over other tutorial genres.

Ensuring that software systems are accessible to users with disabilities is historically neglected but increasingly important for professional software developers. It is imperative that students are familiar with accessible practices to support this often-overlooked form of diversity. We suggest that including accessibility topics when teaching user-interface development skills is a low-effort task that can directly support teaching core software development principles such as "separation of concerns" and "standards compliance." In this lightning talk we describe our initial efforts to integrate accessibility and accessible design as "first-class" topics into our department's required course on web development, including specific examples of concepts covered, classroom activities, and assignments. We also discuss suggestions for how to potentially integrate accessibility topics into other computer science courses which include any kinds of front-end user interfaces. The goal of this talk is to promote awareness of accessibility concerns, demonstrate the ease by which educators can include such material, and encourage discussion about how to engage students in such diversity considerations throughout the curriculum.

Learners regularly abandon online coding tutorials when they get bored or frustrated, but there are few techniques for anticipating this abandonment to intervene. In this paper, we examine the feasibility of predicting abandonment with machine-learned classifiers. Using interaction logs from an online programming game, we extracted a collection of features that are potentially related to learner abandonment and engagement, then developed classifiers for each level. Across the first five levels of the game, our classifiers successfully predicted 61% to 76% of learners who did not complete the next level, achieving an average AUC of 0.68. In these classifiers, features negatively associated with abandonment included account activation and help-seeking behaviors, whereas features positively associated with abandonment included features indicating difficulty and dis-engagement. These findings highlight the feasibility of providing timely intervention to learners likely to quit.

More people are learning to code than ever, but most learning opportunities do not explicitly teach the problem solving skills necessary to succeed at open-ended programming problems. In this paper, we present a new approach to impart these skills, consisting of: 1) explicit instruction on programming problem solving, which frames coding as a process of translating mental representations of problems and solutions into source code, 2) a method of visualizing and monitoring progression through six problem solving stages, 3) explicit, on-demand prompts for learners to reflect on their strategies when seeking help from instructors, and 4) context-sensitive help embedded in a code editor that reinforces the problem solving instruction. We experimentally evaluated the effects of our intervention across two 2-week web development summer camps with 48 high school students, finding that the intervention increased productivity, independence, programming self-efficacy, metacognitive awareness, and growth mindset. We discuss the implications of these results on learning technologies and classroom instruction.

While prior work has investigated many aspects of programming problem solving, the role of self-regulation in problem solving success has received little attention. In this paper we contribute a framework for reasoning about self-regulation in programming problem solving. We then use this framework to investigate how 37 novice programmers of varying experience used self-regulation during a sequence of programming problems. We analyzed the extent to which novices engaged in five kinds of self-regulation during their problem solving, how this self-regulation varied between students enrolled in CS1 and CS2, and how self-regulation played a role in structuring problem solving. We then investigated the relationship between self-regulation and programming errors. Our results indicate that while most novices engage in self-regulation to navigate and inform their problem solving efforts, these self-regulation efforts are only effective when accompanied by programming knowledge adequate to succeed at solving a given problem, and only some types of self-regulation appeared related to errors. We discuss the implications of these findings on problem solving pedagogy in computing education.

Presents an editorial discusses the current status and activities supported by this publication.

As we increasingly strive for scientific rigor and generalizability in HCI research, should we entertain any hope that by doing good science, our discoveries will eventually be more transferrable to industry? We present an in-depth case study of how an HCI research innovation goes through the process of transitioning from a university project to a revenue-generating startup financed by venture capital. The innovation is a novel contextual help system for the Web, and we reflect on the different methods used to evaluate it and how research insights endure attempted dissemination as a commercial product. Although the extent to which any innovation succeeds commercially depends on a number of factors like market forces, we found that our HCI innovation with user-centered origins was in a unique position to gain traction with customers and garner buy-in from investors. However, since end users were not the buyers of our product, a strong user-centered focus obfuscated other critical needs of the startup and pushed out perspectives of non-user-centered stakeholders. To make the research-to-product transition, we had to focus on adoption-centered design, the process of understanding and designing for adopters and stakeholders of the product. Our case study raises questions about how we evaluate the novelty and research contributions of HCI innovations with respect to their potential for commercial impact.

People are increasingly turning to online resources to learn to code. However, despite their prevalence, it is still unclear how successful these resources are at teaching CS1 programming concepts. Using a pretest-posttest study design, we measured the performance of 60 novices before and after they used one of the following, randomly assigned learning activities: 1) complete a Python course on a website called Codecademy, 2) play through and finish a debugging game called Gidget, or 3) use Gidget's puzzle designer to write programs from scratch. The pre- and post-test exams consisted of 24 multiple choice questions that were selected and validated based on data from 1,494 crowdsourced respondents. All 60 of our novices across the three conditions did poorly on the exams overall in both the pre-tests and post-tests (e.g., the best median post-test score was 50% correct). However, those completing the Codecademy course and those playing through the Gidget game showed over a 100% increase in correct answers when comparing their post-test exam scores to their pre-test exam scores. Those playing Gidget, however, achieved these same learning gains in half the time. This was in contrast to novices that used the puzzle designer, who did not show any measurable learning gains. All participants performed similarly within their own conditions, regardless of gender, age, or education. These findings suggest that discretionary online educational technologies can successfully teach novices introductory programming concepts (to a degree) within a few hours when explicitly guided by a curriculum.

Good software engineers are essential to the creation of good software. However, most of what we know about software-engineering expertise are vague stereotypes, such as 'excellent communicators' and 'great teammates'. The lack of specificity in our understanding hinders researchers from reasoning about them, employers from identifying them, and young engineers from becoming them. Our understanding also lacks breadth: what are all the distinguishing attributes of great engineers (technical expertise and beyond)? We took a first step in addressing these gaps by interviewing 59 experienced engineers across 13 divisions at Microsoft, uncovering 53 attributes of great engineers. We explain the attributes and examine how the most salient of these impact projects and teams. We discuss implications of this knowledge on research and the hiring and training of engineers.

Web developers often want to repurpose interactive behaviors from third-party web pages, but struggle to locate the specific source code that implements the behavior. This task is challenging because developers must find and connect all of the non-local interactions between event-based JavaScript code, declarative CSS styles, and web page content that combine to express the behavior.

Many systems are designed to help novices who want to learn programming, but few support those who are not interested in learning (more) programming. This paper targets the subset of end-user programmers (EUPs) in this category. We present a set of principles on how to help EUPs like this learn just a little when they need to overcome a barrier. We then instantiate the principles in a prototype and empirically investigate the principles in two studies: a formative think-aloud study and a pair of summer camps attended by 42 teens. Among the surprising results were the complementary roles of implicitly actionable hints versus explicitly actionable hints, and the importance of both context-free and context-sensitive availability. Under these principles, the camp participants required significantly less in-person help than in a previous camp to learn the same amount of material in the same amount of time.

Computer programming is now used broadly across many industries, with a diversity of working adults writing programs and interacting with code as part of their jobs. However, negative attitudes toward programming continue to deter many from studying computer science and pursuing careers in technology. To begin understanding adults' attitudes toward computer programming and how we can improve them, we used an educational video game to give 200 adult participants a concrete programming experience via the web, and then collected their self-reported opinions about programming. We found that adults initially had poor attitudes toward programming, believing that it was difficult, boring, and something they generally could not learn. After the online learning experience, their attitudes improved significantly, regardless of gender, population density, or level of education. These results demonstrate that adult attitudes toward programming, while initially negative, can be quickly changed with a brief, positive exposure to programming.

Although there are many systems designed to engage people in programming, few explicitly teach the subject, expecting learners to acquire the necessary skills on their own as they create programs from scratch. We present a principled approach to teach programming using a debugging game called Gidget, which was created using a unique set of seven design principles. A total of 44 teens played it via a lab study and two summer camps. Principle by principle, the results revealed strengths, problems, and open questions for the seven principles. Taken together, the results were very encouraging: learners were able to program with conditionals, loops, and other programming concepts after using the game for just 5 hours.

Online games have the potential to reach a wide audience and teach new skills. I propose to use Gidget, an online debugging game, to teach novices computer programming concepts in an engaging way. Learners must debug faulty programs to progress through the game, which are set up in modules to teach specific computer programming concepts. Once all the levels are complete, learners are given the option to further engage in the game by creating their own levels that can be shared with their friends and family. Over 800 people have played the game online as part of several research studies and it will be released freely to the public in the near future.

We present a multi-site field study to evaluate LemonAid, a crowdsourced contextual help approach that allows users to retrieve relevant questions and answers by making selections within the interface. We deployed LemonAid on 4 different web sites used by thousands of users and collected data over several weeks, gathering over 1,200 usage logs, 168 exit surveys, and 36 one-on-one interviews. Our results indicate that over 70% of users found LemonAid to be helpful, intuitive, and desirable for reuse. Software teams found LemonAid easy to integrate with their sites and found the analytics data aggregated by LemonAid a novel way of learning about users' popular questions. Our work provides the first holistic picture of the adoption and use of a crowdsourced contextual help system and offers several insights into the social and organizational dimensions of implementing such help systems for real-world applications.

Assessments have been shown to have positive effects on learning in compulsory educational settings. However, much less is known about their effects in discretionary learning settings, especially in computing education and educational games. We hypothesized that adding assessments to an educational computing game would provide extra opportunities for players to practice and correct misconceptions, thereby affecting their performance on subsequent levels and their motivation to continue playing. To test this, we designed a game called Gidget, in which players help a robot find and fix defects in programs that follow a mastery learning paradigm. Across two studies, we manipulated the inclusion of multiple choice and self-explanation assessment levels in the game, measuring their impact on engagement and level completion speed. In our first study, we found that including assessments caused learners to voluntarily play longer and complete more levels, suggesting increased engagement; in our second study, we found that including assessments caused learners to complete levels faster, suggesting increased understanding. These findings suggest that including assessments in a discretionary computing education game may be a key design strategy for improving informal learning of computing concepts.

During debugging, a developer must repeatedly and manually reproduce faulty behavior in order to inspect different facets of the program's execution. Existing tools for reproducing such behaviors prevent the use of debugging aids such as breakpoints and logging, and are not designed for interactive, random-access exploration of recorded behavior. This paper presents Timelapse, a tool for quickly recording, reproducing, and debugging interactive behaviors in web applications. Developers can use Timelapse to browse, visualize, and seek within recorded program executions while simultaneously using familiar debugging tools such as breakpoints and logging. Testers and end-users can use Timelapse to demonstrate failures in situ and share recorded behaviors with developers, improving bug report quality by obviating the need for detailed reproduction steps. Timelapse is built on Dolos, a novel record/replay infrastructure that ensures deterministic execution by capturing and reusing program inputs both from the user and from external sources such as the network. Dolos introduces negligible overhead and does not interfere with breakpoints and logging. In a small user evaluation, participants used Timelapse to accelerate existing reproduction activities, but were not significantly faster or more successful in completing the larger tasks at hand. Together, the Dolos infrastructure and Timelapse developer tool support systematic bug reporting and debugging practices.

Web-based technical support such as discussion forums and social networking sites have been successful at ensuring that most technical support questions eventually receive helpful answers. Unfortunately, finding these answers is still quite difficult, since users' textual queries are often incomplete, imprecise, or use different vocabularies to describe the same problem. We present LemonAid, a new approach to help that allows users to find help by instead selecting a label, widget, link, image or other user interface (UI) element that they believe is relevant to their problem. LemonAid uses this selection to retrieve previously asked questions and their corresponding answers. The key insight that makes LemonAid work is that users tend to make similar selections in the interface for similar help needs and different selections for different help needs. Our initial evaluation shows that across a corpus of dozens of tasks and thousands of requests, LemonAid retrieved a result for 90% of help requests based on UI selections and, of those, over half had relevant matches in the top 2 results.

A lack of support for active listening undermines discussion and deliberation on the web. We contribute a design frame identifying potential improvements to web discussion were listening more explicitly encouraged in interfaces. We explore these concepts through a novel interface, Reflect, that creates a space next to every comment where others can summarize the points they hear the commenter making. Deployments on Slashdot, Wikimedia's Strategic Planning Initiative, and a local civic effort suggest that interfaces for listening may have traction for general use on the web.

Engagement is a necessary condition for learning, and previous studies have shown that engagement can be significantly affected by changing the presentation of game elements within an educational game. In a three condition controlled experiment, we examined how changing the presentation of the data elements referred to in a game's goals would influence the purposefulness of the goals and thereby affect players' motivation to achieve them. A total of 121 self-described programming novices were recruited online to play the game. We found that 1) those using vertebrate elements completed twice the number of levels compared to those using inanimate elements, 2) those using vertebrate and invertebrate elements spent significantly more time playing the game overall compared to those using inanimate elements, and 3) those using inanimate elements were more likely to quit the game, especially on difficult levels. These findings suggest that the presentation of game elements that influence the purposefulness of goals can play a significant role in keeping self-guided learners engaged in learning tasks.

Despite the growing research on usability in the pre-development phase, we know little about post-deployment usability activities. To characterize these activities, we surveyed 333 full-time usability professionals and consultants working in large and small corporations from a wide range of industries. Our results show that, as a whole, usability professionals are currently not playing a substantial role in the post-deployment phase compared to other phases of user-centered design, but when they do, practitioners find their interactions quite valuable. We highlight opportunities in HCI research and practice to bridge this gap by working more closely with software support and maintenance teams. We also raise the need to understand what might be called 'usability maintenance,' that is, the process and procedures, by which usability is maintained after deployment.

While usability methods such as user studies and inspections can reveal a wide range of problems, they do so for only a subset of an application's features and states. We present FeedLack, a tool that explores the full range of web applications' behaviors for one class of usability problems, namely that of missing feedback. It does this by enumerating control flow paths originating from user input, identifying paths that lack output-affecting code. FeedLack was applied to 330 applications; of the 129 that contained input handlers and did not contain syntax errors, 115 were successfully analyzed, resulting in 647 warnings. Of these 36% were missing crucial feedback; 34% were executable and missing feedback, but followed conventions that made feedback inessential; 18% were scenarios that did produce feedback; 12% could not be executed. We end with a discussion of the viability of FeedLack as a usability testing tool.

Many novice programmers view programming tools as all-knowing, infallible authorities about what is right and wrong about code. This misconception is particularly detrimental to beginners, who may view the cold, terse, and often judgmental errors from compilers as a sign of personal failure. It is possible, however, that attributing this failure to the computer, rather than the learner, may improve learners' motivation to program. To test this hypothesis, we present Gidget, a game where the eponymous robot protagonist is cast as a fallible character that blames itself for not being able to correctly write code to complete its missions. Players learn programming by working with Gidget to debug its problematic code. In a two-condition controlled experiment, we manipulated Gidget's level of personification in: communication style, sound effects, and image. We tested our game with 116 self-described novice programmers recruited on Amazon's Mechanical Turk and found that, when given the option to quit at any time, those in the experimental condition (with a personable Gidget) completed significantly more levels in a similar amount of time. Participants in the control and experimental groups played the game for an average time of 39.4 minutes (SD=34.3) and 50.1 minutes (SD=42.6) respectively. These finding suggest that how programming tool feedback is portrayed to learners can have a significant impact on motivation to program and learning success.

Many software producers utilize beta programs to predict post-release quality and to ensure that their products meet quality expectations of users. Prior work indicates that software producers need to adjust predictions to account for usage environments and usage scenarios differences between beta populations and post-release populations. However, little is known about how usage characteristics relate to field quality and how usage characteristics differ between beta and post-release. In this study, we examine application crash, application hang, system crash, and usage information from millions of Windows® users to 1) examine the effects of usage characteristics differences on field quality (e.g. which usage characteristics impact quality), 2) examine usage characteristics differences between beta and post-release (e.g. do impactful usage characteristics differ), and 3) report experiences adjusting field quality predictions for Windows. Among the 18 usage characteristics that we examined, the five most important were: the number of application executed, whether the machines was pre-installed by the original equipment manufacturer, two sub-populations (two language/geographic locales), and whether Windows was 64-bit (not 32-bit). We found each of these usage characteristics to differ between beta and post-release, and by adjusting for the differences, accuracy of field quality predictions for Windows improved by ~59%.

While many studies have investigated the challenges that developers face in finding and using API documentation, few have considered the role of developers' conceptual knowledge in these tasks. We designed a study in which developers were asked to explore the feasibility of two requirements concerning networking protocols and application platforms that most participants were unfamiliar with, observing the effect that a lack of conceptual knowledge had on their use of documentation. Our results show that without conceptual knowledge, developers struggled to formulate effective queries and to evaluate the relevance or meaning of content they found. Our results suggest that API documentation should not only include detailed examples of API use, but also thorough introductions to the concepts, standards, and ideas manifested in an API's data structures and functionality.

Machine learning techniques are increasingly used in intelligent assistants , that is, software targeted at and continuously adapting to assist end users with email, shopping, and other tasks. Examples include desktop SPAM filters, recommender systems, and handwriting recognition. Fixing such intelligent assistants when they learn incorrect behavior, however, has received only limited attention. To directly support end-user “debugging” of assistant behaviors learned via statistical machine learning, we present a Why-oriented approach which allows users to ask questions about how the assistant made its predictions, provides answers to these “why” questions, and allows users to interactively change these answers to debug the assistant's current and future predictions. To understand the strengths and weaknesses of this approach, we then conducted an exploratory study to investigate barriers that participants could encounter when debugging an intelligent assistant using our approach, and the information those participants requested to overcome these barriers. To help ensure the inclusiveness of our approach, we also explored how gender differences played a role in understanding barriers and information needs. We then used these results to consider opportunities for Why-oriented approaches to address user barriers and information needs.

Although usability methods are widely used for evaluating conventional graphical user interfaces and websites, there is a growing concern that current approaches are inadequate for evaluating complex, domain-specific tools. We interviewed 21 experienced usability professionals, including in-house experts, external consultants, and managers working in a variety of complex domains, and uncovered the challenges commonly posed by domain complexity and how practitioners work around them. We found that despite the best efforts by usability professionals to get familiar with complex domains on their own, the lack of formal domain expertise can be a significant hurdle for carrying out effective usability evaluations. Partnerships with domain experts lead to effective results as long as domain experts are willing to be an integral part of the usability team. These findings suggest that for achieving usability in complex domains, some fundamental educational changes may be needed in the training of usability professionals.

Many power users that contribute to open source projects have no intention of becoming regular contributors; they just want a bug fixed or a feature implemented. How often do these users participate in open source projects and what do they contribute? To investigate these questions, we analyzed the reports of Mozilla contributors who reported problems but were never assigned problems to fix. These analyses revealed that over 11 years and millions of reports, most of these 150,000 users reported non-issues that devolved into technical support, redundant reports with little new information, or narrow, expert feature requests. Reports that did lead to changes were reported by a comparably small group of experienced, frequent reporters, mostly before the release of Firefox 1. These results suggest that the primary value of open bug reporting is in recruiting talented reporters, and not in deriving value from the masses.

We present Gestalt, a development environment designed to support the process of applying machine learning. While traditional programming environments focus on source code, we explicitly support both code and data. Gestalt allows developers to implement a classification pipeline, analyze data as it moves through that pipeline, and easily transition between implementation and analysis. An experiment shows this significantly improves the ability of developers to find and fix bugs in machine learning systems. Our discussion of Gestalt and our experimental observations provide new insight into general-purpose support for the machine learning process.

Open bug reporting allows end-users to express a vast array of unwanted software behaviors. However, users’ expectations often clash with developers’ implementation intents. We created a classification of seven common expectation violations cited by end-users in bug report descriptions and applied it to 1,000 bug reports from the Mozilla project. Our results show that users largely described bugs as violations of their own personal expectations, of specifications, or of the user community’s expectations. We found a correlation between a reporter’s expression of which expectation was being violated and whether or not the bug would eventually be fixed. Specifically, when bugs were expressed as violations of community expectations rather than personal expectations, they had a better chance of being fixed.

Many dynamic programming language features, such as implicit declaration, reflection, and code generation, make it difficult to verify the existence of identifiers through standard program analysis. We present an alternative verification, which, rather than analyzing the semantics of code, highlights any name or pair of names that appear only once across a program's source files. This uniqueness heuristic is implemented for HTML, CSS, and JavaScript, in an interactive editor called Cleanroom, which highlights lone identifiers after each keystroke. Through an online experiment, we show that Cleanroom detects real errors, that it helps developers find these errors more quickly than developers can find them on their own, and that this helps developers avoid costly debugging effort by reducing how many times a program is executed with potential errors. The simplicity and power of Cleanroom's heuristic may generalize well to other dynamic languages with little support for edit-time name verification.

When software developers want to understand the reason for a program's behavior, they must translate their questions about the behavior into a series of questions about code, speculating about the causes in the process. The Whyline is a new kind of debugging tool that avoids such speculation by instead enabling developers to select a question about program output from a set of “why did and why didn't” questions extracted from the program's code and execution. The tool then finds one or more possible explanations for the output in question. These explanations are derived using a static and dynamic slicing, precise call graphs, reachability analyses, and new algorithms for determining potential sources of values. Evaluations of the tool on two debugging tasks showed that developers with the Whyline were three times more successful and twice as fast at debugging, compared to developers with traditional breakpoint debuggers. The tool has the potential to simplify debugging and program understanding in many software development contexts.

Debugging and diagnostic tools are some of the most important software development tools, but most expect developers choose the right code to inspect. Unfortunately, this rarely occurs. A new tool called the Whyline is described which avoids such speculation by allowing developers to select questions about a program's output. The tool then helps developers work backwards from output to its causes. The prototype, which supports Java programs, was evaluated in an experiment in which participants investigated two real bug reports from an open source project using either the Whyline or a breakpoint debugger. Whyline users were successful about three times as often and about twice as fast compared to the control group, and were extremely positive about the tool's ability to simplify diagnostic tasks in software development work.

The results of a machine learning from user behavior can be thought of as a program, and like all programs, it may need to be debugged. Providing ways for the user to debug it matters, because without the ability to fix errors users may find that the learned program's errors are too damaging for them to be able to trust such programs. We present a new approach to enable end users to debug a learned program. We then use an early prototype of our new approach to conduct a formative study to determine where and when debugging issues arise, both in general and also separately for males and females. The results suggest opportunities to make machine-learned programs more effective tools.

Little is known about the formation of people's first perceptions about computers and computer code, yet it is likely that these impressions have a lasting effect on peoples' use of technology in their lives and careers. Brief autobiographical essays about these first impressions were solicited from a diverse population of young adults and these essays were analyzed for factors that contributed to positive and negative attitudes about technology, formation of self-efficacy, and authors' relationship with computing later in life. The results suggest that first encounters with code must be accessible, error-tolerant and socially engaging, that mentorship can be a crucial factor in the acquisition of programming skills, and that cultivating positive self-efficacy in programming skills requires repeated positive exposure across the lifespan. These results raise several issues for novice programming languages and tools and suggest a number of new approaches to computing education.

The seventh annual graduate consortium at VL/HCC addresses the question: How can researchers and designers of computational tools support problem solving and information manipulation by diverse user populations? For the seventh year, the U.S. National Science Foundation continues its sponsorship of a graduate student consortium at the VL/HCC Symposia (NSF grant # IIS-0929989). These workshops, on themes related to diversity and universal access of software development technologies, have brought together graduate students, faculty mentors, and conference attendees to discuss the innovative work of the students and provide feedback and suggestions on their research. The events have produced considerable excitement and community building around approaches for reaching broad populations.

When software developers want to understand the reason for a program's behavior, they must translate their questions about the behavior into a series of questions about code, speculating about the causes in the process. The Whyline is a new kind of debugging tool that avoids such speculation by instead enabling developers to select a question about program output from a set of why did and why didn't questions derived from the program's code and execution. The tool then finds one or more possible explanations for the output in question, using a combination of static and dynamic slicing, precise call graphs, and new algorithms for determining potential sources of values and explanations for why a line of code was not reached. Evaluations of the tool on one task showed that novice programmers with the Whyline were twice as fast as expert programmers without it. The tool has the potential to simplify debugging in many software development contexts.

Designers are skilled at sketching and prototyping the look of interfaces, but to explore various behaviors (what the interface does in response to input) typically requires programming using Javascript, ActionScript for Flash, or other languages. In our survey of 259 designers, 86% reported that the behavior is more difficult to prototype than the appearance. Often (78% of the time), designing the behavior requires collaborating with developers, but 76% of designers reported that communicatin1g the behavior to developers was more difficult than the appearance. Other results include that annotations such as arrows and paragraphs of text are used on top of sketches and storyboards to explain behaviors, and designers want to explore multiple versions of behaviors, but today’s tools make this difficult. The results provide new ideas for future tools.

While a designer's focus used to be the design of non-interactive elements such as graphics or animations, today's designers deal with various levels of interactivity such as mouse, keyboard and touch screen interaction. Unfortunately, it is challenging for designers to create these diverse interactions since most implementation tools such as Flash require the use of conventional programming languages and do not support the natural expressions used by designers. To better understand how designers think about interactive behaviors, we conducted a lab study where designers and programmers described various primitive and composite interactive behaviors using their own language. From this, we learned that there is significant commonality among designers in terms of the verbs, syntax, and structure when describing interactivity. These results can help guide the way to building more natural programming languages and environments for designers to facilitate the development of interactive behaviors.

Software developers are rooted in the written form of their code, yet they often draw diagrams representing their code. Unfortunately, we still know little about how and why they create these diagrams, and so there is little research to inform the design of visual tools to support developers' work. This paper presents findings from semi-structured interviews that have been validated with a structured survey. Results show that most of the diagrams had a transient nature because of the high cost of changing whiteboard sketches to electronic renderings. Diagrams that documented design decisions were often externalized in these temporary drawings and then subsequently lost. Current visualization tools and the software development practices that we observed do not solve these issues, but these results suggest several directions for future research.

Previous research has documented the fragmented nature of software development work. To explain this in more detail, we analyzed software developers' day-to-day information needs. We observed seventeen developers at a large software company and transcribed their activities in go-minute sessions. We analyzed these logs for the information that developers sought, the sources that they used, and the situations th at prevented inform action from being acquired. We identified twenty-one information types and cataloged the outcome and source when each type of information was sought. The most frequently sought information included awareness about artifacts and coworkers. The most often deferred searches included knowledge about design and program behavior, such as why code was written a particular way, what a program was supposed to do, and the cause of a program state. Developers often had to defer tasks because the only source of knowledge was unavailable coworkers.

Recent advances in programming environments have focused on improving programmer productivity by utilizing the inherent structure in computer programs. However, because these environments represent code as plain text, it is difficult and sometimes impossible to embed interactive tools, annotations, and alternative views in the code itself. Barista is an implementation framework that enables the creation of such user interfaces by simplifying the implementation of editors that represent code internally as an abstract syntax tree and maintain a corresponding, fully structured visual representation on-screen. Barista also provides designers of editors with a standard text-editing interaction technique that closely mimics that of conventional text editors, overcoming a central usability issue of previous structured code editors.

Modern applications such as Microsoft Word have many automatic features and hidden dependencies that are frequently helpful but can be mysterious to both novice and expert users. The ""Crystal"" application framework provides an architecture and interaction techniques that allow programmers to create applications that let the user ask a wide variety of questions about why things did and did not happen, and how to use the related features of the application without using natural language. A user can point to an object or a blank space and get a popup list of questions about it, or the user can ask about recent actions from a temporal list. Parts of a text editor were implemented to show that these techniques are feasible, and a user test suggests that they are helpful and well-liked.

One reason debugging is the most time-consuming part of software development is because developers struggle to map their questions about a program's behavior onto debugging tools' limited support for analyzing code. Interrogative debugging is a new debugging paradigm that allows developers to ask questions directly about their programs' output, helping them to more efficiently and accurately determine what parts of the system to understand. An interrogative debugging prototype called the Whyline is described, which has been shown to reduce debugging time by a factor of eight. Several extensions and generalizations to it are proposed, including plans for evaluating their effectiveness.

There is little understanding of how people describe software problems, but a variety of tools solicit, manage, and analyze these descriptions in order to streamline software development. To inform the design of these tools and generate ideas for new ones, an study of nearly 200,000 bug report titles was performed. The titles of the reports generally described a software entity or behavior, its inadequacy, and an execution context, suggesting new designs for more structured report forms. About 95% of noun phrases referred to visible software entities, physical devices, or user actions, suggesting the feasibility of allowing users to select these entities in debuggers and other tools. Also, the structure of the titles exhibited sufficient regularity to parse with an accuracy of 89%, enabling a number of new automated analyses. These findings and others have many implications for tool design and software engineering

Information workers such as administrative staff, consultants, and their managers constitute one of the largest groups of end users, yet little research about their usage of programming features is available to guide development of end user programming tools. In this paper, we describe our survey of over 800 information workers and our analysis of their feature usage in applications such as spreadsheets, browsers, and databases. Our factor analysis reveals three clusters of features - macro features, linked structure features, and imperative features - such that information workers with an inclination to use a feature in each cluster also were inclined to use other features in that cluster, even though each cluster spans several tools. We discuss the implications for research aimed at providing end user programming tools for information workers

Much of software developers' time is spent understanding unfamiliar code. To better understand how developers gain this understanding and how software development environments might be involved, a study was performed in which developers were given an unfamiliar program and asked to work on two debugging tasks and three enhancement tasks for 70 minutes. The study found that developers interleaved three activities. They began by searching for relevant code both manually and using search tools; however, they based their searches on limited and misrepresentative cues in the code, environment, and executing program, often leading to failed searches. When developers found relevant code, they followed its incoming and outgoing dependencies, often returning to it and navigating its other dependencies; while doing so, however, Eclipse's navigational tools caused significant overhead. Developers collected code and other information that they believed would be necessary to edit, duplicate, or otherwise refer to later by encoding it in the interactive state of Eclipse's package explorer, file tabs, and scroll bars. However, developers lost track of relevant code as these interfaces were used for other tasks, and developers were forced to find it again. These issues caused developers to spend, on average, 35 percent of their time performing the mechanics of navigation within and between source files. These observations suggest a new model of program understanding grounded in theories of information foraging and suggest ideas for tools that help developers seek, relate, and collect information in a more effective and explicit manner.

The computer and communication systems that office workers currently use tend to interrupt at inappropriate times or unduly demand attention because they have no way to determine when an interruption is appropriate. Sensor?based statistical models of human interruptibility offer a potential solution to this problem. Prior work to examine such models has primarily reported results related to social engagement, but it seems that task engagement is also important. Using an approach developed in our prior work on sensor?based statistical models of human interruptibility, we examine task engagement by studying programmers working on a realistic programming task. After examining many potential sensors, we implement a system to log low?level input events in a development environment. We then automatically extract features from these low?level event logs and build a statistical model of interruptibility. By correctly identifying situations in which programmers are non?interruptible and minimizing cases where the model incorrectly estimates that a programmer is non?interruptible, we can support a reduction in costly interruptions while still allowing systems to convey notifications in a timely manner.

Recently, several innovative tools have found their way into mainstream use in modern development environments. However, most of these tools have focused on creating and modifying code, despite evidence that most of programmers' time is spent understanding code as part of maintenance tasks. If new tools were designed to directly support these maintenance tasks, what types would be most helpful? To find out, a study of expert Java programmers using Eclipse was performed. The study suggests that maintenance work consists of three activities: (1) forming a working set of task-relevant code fragments; (2) navigating the dependencies within this working set; and (3) repairing or creating the necessary code. The study identified several trends in these activities, as well as many opportunities for new tools that could save programmers up to 35% of the time they currently spend on maintenance tasks.

Direct-manipulation editors for structured data are increasingly common. While such editors can greatly simplify the creation of structured data, there are few tools to simplify the creation of the editors themselves. This paper presents Citrus, a new programming language and user interface toolkit designed for this purpose. Citrus offers language-level support for constraints, restrictions and change notifications on primitive and aggregate data, mechanisms for automatically creating, removing, and reusing views as data changes, a library of widgets, layouts and behaviors for defining interactive views, and two comprehensive interactive editors as an interface to the language and toolkit itself. Together, these features support the creation of editors for a large class of data and code.

There are many common spreadsheet errors that traditional spreadsheet systems do not help users find. This paper presents a statically-typed spreadsheet language that adds additional information about the objects that the spreadsheet values represent. By annotating values with both units and labels, users denote both the system of measurement in which the values are expressed as well as the properties of the objects to which the values refer. This information is used during computation to detect some invalid computations and allow users to identify properties of the resulting values.

Debugging is still among the most common and costly of programming activities. One reason is that current debugging tools do not directly support the inquisitive nature of the activity. Interrogative Debugging is a new debugging paradigm in which programmers can ask why did and even why didn't questions directly about their program's runtime failures. The Whyline is a prototype Interrogative Debugging interface for the Alice programming environment that visualizes answers in terms of runtime events directly relevant to a programmer's question. Comparisons of identical debugging scenarios from user tests with and without the Whyline showed that the Whyline reduced debugging time by nearly a factor of 8, and helped programmers complete 40% more tasks.

An important part of helping learners detect, repair and avoid software errors is providing semantic support for learners while they manipulate their code. Unfortunately, usability aspects of both textual and direct-manipulation environments limit their ability to provide such support. Preliminary findings from exploratory studies are discussed, and several design requirements for a more flexible and supportive programming environment are identified.

As programming skills increase in demand and utility, the learnability of end-user programming systems is of utmost importance. However, research on learning barriers in programming systems has primarily focused on languages, overlooking potential barriers in the environment and accompanying libraries. To address this, a study of beginning programmers learning Visual Basic.NET was performed. This identified six types of barriers: design, selection, coordination, use, understanding, and information. These barriers inspire a new metaphor of computation, which provides a more learner-centric view of programming system design.