VLDB 2026 Research / reviewers in the wild / expert
Andreas Stefik
dblp:06/1152
· DBLP profile ↗
47ranked-venue papers
14as first author
13since 2021 · last 2026
0000-0002-2554-5862ORCID · verified
Domains — the database's venue-derived domains; a paper can count in several
Human-computer interaction and ubiquitous computing · 27 · 6 first-author · 13 since 2021Software engineering, systems software and programming languages · 16 · 7 first-authorApplied, interdisciplinary, general and emerging computing · 3 · 1 first-authorArtificial intelligence and machine learning · 1
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2026 | Accessible Keyboard Controls for Block Ordering Problems
Peter Fowles, Andreas Stefik, Brianna Blaser, David H. Smith, Seth Poulsen |
ITiCSE (1) | 2 |
| 2026 | Disability and Accessibility in Computer Science EducationabstractStudents with disabilities face a variety of challenges in computer science education including those related to inaccessible curriculum, instruction and tools. In addition, few computer science classes teach students to understand accessibility and design accessible technology. This BOF will bring together individuals who are interested in increasing the accessibility of computing education as well as those interested in teaching about accessibility. Participants will share strategies to help each other do a better job of addressing accessibility in our classes and research projects. Resources related to accessible tools and instruction, universal design of learning, opportunities for students, teaching accessibility, and more will be shared. Brianna Blaser, Maya Cakmak, Richard E. Ladner, Amy J. Ko, Andreas Stefik, Raja S. Kushalnagar, Stacy M. Branham |
SIGCSE (2) | 5 |
| 2026 | Students with Disabilities in Computer Science Principles: An Examination of Capacity, Access, and ParticipationabstractAmong ongoing efforts to broaden participation in K–12 computer science (CS) education, the Advanced Placement (AP) Computer Science Principles (CSP) course receives a lot of attention. While prior research has shown increased participation among some historically underrepresented groups, little is known about how the course serves students with disabilities. Sara B. Frey, Hannah Williams, Andreas Stefik |
SIGCSE (1) | 3 |
| 2026 | Toward Accessible Parsons Problems on Mobile PlatformsabstractParsons problems have become a mainstay of computer science education. They are heavily used among students, especially in K-12 and provide a small puzzle-like experience for students to practice their skills. Today, while prior work has explored complex issues with accessibility and block languages in general, the 2024 changes to accessibility regulations by the U.S. Department of Justice includes new rules around mobile platforms. These rules are ill-defined and in need of evaluation. In this work, we make several contributions. First, we conducted an evaluation of existing blocks with respect to their regulatory compliance and discuss a new blocks technology that we developed that meets these new mobile guidelines. Second, we conducted three empirical studies using Parsons problems to evaluate the usability of the technology with teachers of the visually impaired (n = 32), high-school students with diverse disabilities (n = 28), and high-school students with blindness or low vision (n = 13). Timothy Kluthe, Gabriel Contreras, William Allee, Wilfredo Joshua Robinson Moore, Namrata Roy, Hannah Williams, Alex Hoffman, Derrick W. Smith, Brianna Blaser, Jenna L. Gorlewicz, Nicholas A. Giudice, Andreas Stefik |
SIGCSE (1) | 12 |
| 2025 | Disability and Accessibility in Computer Science EducationabstractStudents with disabilities face a variety of challenges in computer science education including those related to stigma around disability, inaccessible curriculum, instruction and tools, disability disclosure, and a lack of mentors. In addition, few computer science classes teach students to understand accessibility and design accessible technology. This BOF will bring together individuals who are interested in increasing the representation of people with disabilities in computing and improving their success as well as those interested in teaching about accessibility. Participants will share strategies to help each other do a better job of addressing disability inclusion and accessibility in our classes and research projects. Resources related to accessible tools and instruction, universal design of learning, opportunities for students, teaching accessibility, and more will be shared. Brianna Blaser, Maya Cakmak, Richard E. Ladner, Andreas Stefik, Raja S. Kushalnagar, Stacy M. Branham, Amy J. Ko |
SIGCSE (2) | 4 |
| 2025 | The Implication of Accessibility and Privacy Legislation on Classroom Programming ToolsabstractData privacy and software accessibility are important issues. In order to enforce correct practice, many American states (and other countries) are introducing legislation to enforce data privacy and software accessibility for tools that are used in school classrooms. This has widespread implications for the use of programming tools in computer science classrooms in the USA and globally. The makers of such tools must ensure compliance with these laws and typically school districts are requiring certification of this before teachers can use them. This applies even if the tool is making no direct or indirect revenue, which adds a technical and legal burden even for free open-source tools. This birds-of-a-feather session is intended to bring together everyone affected by these changes -- teachers, school districts, policy-makers, and programming tool makers -- to discuss the ramifications of these issues and form ideas for best practices. Neil Brown 0001, Samantha Schwartz, Andreas Stefik |
SIGCSE (2) | 3 |
| 2024 | Disability and Accessibility in Computer Science EducationabstractStudents with disabilities face a variety of challenges in computer science education including those related to stigma around disability, inaccessible curriculum, instruction and tools, disability disclosure, and a lack of mentors. In addition, few computer science classes teach students to understand accessibility and design accessible technology. This BOF will bring together individuals who are interested in increasing the representation of people with disabilities in computing and improving their success as well as those interested in teaching about accessibility. Participants will share strategies to help each other do a better job of addressing disability inclusion and accessibility in our classes and research projects. Resources related to accessible tools and instruction, universal design of learning, opportunities for students, teaching accessibility, and more will be shared. Richard E. Ladner, Brianna Blaser, Andreas Stefik, Amy J. Ko, Raja S. Kushalnagar |
SIGCSE (2) | 3 |
| 2024 | Accessible to Whom? Bringing Accessibility to BlocksabstractThe introduction of block-based programming has gradually changed the landscape of programming education, particularly for school children. Block languages today, however, have serious technical barriers to students with disabilities. For example, block languages are generally not screen reader accessible, incompatible with braille, and contain serious problems for users with motor impairments. No student with a disability should ever be denied access to learning computer science and they do not have to be. To help rectify this, we present a new approach to the design of block languages called Quorum Blocks. Quorum Blocks uses a custom hardware accelerated graphical rendering pipeline that takes into account how screen readers and other devices work under the hood. We discuss these technical details and demonstrate that accessibility support can be fully achieved without meaningfully losing either the look of modern blocks or their visual output. We present the results from focus groups that highlight the barriers students faced with a variety of disabilities when using the first version of Quorum Blocks. We focus especially on challenges with low vision users, screen reader users, or those using no mouse and only one hand to type. Block languages built using either our techniques, or on top of our libraries, would become accessible out of the box. Andreas Stefik, William Allee, Gabriel Contreras, Timothy Kluthe, Alex Hoffman, Brianna Blaser, Richard E. Ladner |
SIGCSE (1) | 1 |
| 2023 | Disability in Computer Science EducationabstractStudents with disabilities face a variety of challenges in computer science education including those related to stigma around disability, inaccessible tools and instruction, disability disclosure, and a lack of mentors. This BOF will bring together individuals who are interested in increasing the representation of students with disabilities in computing and improving their success. Participants will share strategies to help each other do a better job of including these students in our classes and research projects. Resources related to accessible tools and instruction, universal design of learning, opportunities for students, and more will be shared. Richard E. Ladner, Brianna Blaser, Andreas Stefik, Amy J. Ko |
SIGCSE (2) | 3 |
| 2022 | Disability in Computer Science EducationabstractStudents with disabilities face a variety of challenges including those related to stigma around disability, inaccessible tools and instruction, disability disclosure, and a lack of mentors. This BOF will bring together individuals who are interested in increasing the representation of students with disabilities in computing and improving their success. Participants will share strategies to help each other do a better job of including these students in our classes and research projects. Resources related to accessible tools and instruction, universal design of learning, opportunities for students, and more will be shared. Richard E. Ladner, Andreas Stefik, Amy J. Ko, Brianna Blaser, Stacy M. Branham, Raja S. Kushalnagar |
SIGCSE (2) | 2 |
| 2021 | Computing Competencies for Undergraduate Data Science Programs: an ACM Task Force Final ReportabstractIn this session, members of the ACM Data Science (DS) Task Force will present the final draft of Computing Competencies for Undergraduate Data Science Programs. Drafting this document has been a three-year process, in which the task force has released preliminary drafts, sought input from the community, and responded to the community's helpful feedback. Our intent is that the session be an exchange that will clarify the contents of the report and provide participants with ways to put the report into practice at their own institutions. This session should be of interest to all SIGCSE attendees, but especially to faculty developing college-level curricula in Data Science. Andrea Pohoreckyj Danyluk, Paul M. Leidig, Andrew D. McGettrick, Lillian N. Cassel, Maureen Doyle, Christian Servin, Karl Schmitt, Andreas Stefik |
SIGCSE | 8 |
| 2021 | Fun and Engaging Pre-CS1 Programming LanguagesabstractThe CSforALL movement to bring computational thinking to K-12 has been a boon for practitioners and language developers. This panel features three educators passionate about a particular lan- guage that has been successful with K-12 audiences. Each will demonstrate their language, describe what makes it unique, and share some of the fun and engaging projects students have created. Dan Garcia 0001, Michael P. Rogers, Andreas Stefik |
SIGCSE | 3 |
| 2021 | Disability in Computer Science EducationabstractStudents with disabilities face a variety of challenges including those related to stigma around disability, inaccessible tools and instruction, disability disclosure, and a lack of mentors. This BOF will bring together individuals who are interested in increasing the representation of students with disabilities in computing and improving their success. Participants will share strategies to help each other do a better job of including these students in our classes and research projects. Resources related to accessible tools and instruction, universal design of learning, opportunities for students, and more will be shared. Richard E. Ladner, Andreas Stefik, Amy J. Ko, Brianna Blaser, Sheryl Burgstahler |
SIGCSE | 2 |
| 2020 | Access to Computing Education for Students with DisabilitiesabstractApproximately 10% of computer science and engineering majors have a disability. Students with disabilities face a variety of challenges including those related to stigma around disability, inaccessible tools and instruction, disability disclosure, and a lack of mentors. This BOF will bring together individuals who are interested in increasing the representation of students with disabilities in computing and improving their success. Participants will share strategies to help each other do a better job of including these students in our classes and research projects. Resources related to accessible tools and instruction, universal design of learning, opportunities for students, and more will be shared. Richard E. Ladner, Andreas Stefik, Amy J. Ko, Brianna Blaser |
SIGCSE | 2 |
| 2020 | A randomized controlled trial on the effects of embedded computer language switchingabstractPolyglot programming, the use of multiple programming languages during the development process, is common practice in modern software development. This study investigates this practice through a randomized controlled trial conducted under the context of database programming. Participants in the study were given coding tasks written in Java and one of three SQL-like embedded languages. One was plain SQL in strings, one was in Java only, and the third was a hybrid embedded language that was closer to the host language. We recorded 109 valid data points. Results showed significant differences in how developers of different experience levels code using polyglot techniques. Notably, less experienced programmers wrote correct programs faster in the hybrid condition (frequent, but less severe, switches), while more experienced developers that already knew both languages performed better in traditional SQL (less frequent but more complete switches). The results indicate that the productivity impact of polyglot programming is complex and experience level dependent. Phillip Merlin Uesbeck, Cole S. Peterson, Bonita Sharif, Andreas Stefik |
ESEC/SIGSOFT FSE | 4 |
| 2020 | GPU Programming Productivity in Different Abstraction Paradigms: A Randomized Controlled Trial Comparing CUDA and ThrustabstractCoprocessor architectures in High Performance Computing are prevalent in today’s scientific computing clusters and require specialized knowledge for proper utilization. Various alternative paradigms for parallel and offload computation exist, but little is known about the human factors impacts of using the different paradigms. With computer science student participants from the University of Nevada, Las Vegas with no previous exposure to Graphics Processing Unit programming, our study compared NVIDIA CUDA C/C++ as a control group and the Thrust library. The designers of Thrust claim their higher level of abstraction enhances programmer productivity. The trial was conducted on 91 participants and was administered through our computerized testing platform. Although the study was narrowly focused on the basic steps of an offloaded computation problem and was not intended to be a comprehensive evaluation of the superiority of one approach or the other, we found evidence that although Thrust was designed for ease of use, the abstractions tended to be confusing to students and in several cases diminished productivity. Specifically, abstractions in Thrust for (i) memory allocation through a C++ Standard Template Library-style vector library call, (ii) memory transfers between the host and Graphics Processing Unit coprocessor through an overloaded assignment operator, and (iii) execution of an offloaded routine through a generic transform library call instead of a CUDA kernel routine all performed either equal to or worse than CUDA. Patrick Daleiden, Andreas Stefik, Phillip Merlin Uesbeck |
ACM Trans. Comput. Educ. | 2 |
| 2020 | Analysis of a Randomized Controlled Trial of Student Performance in Parallel Programming using a New Measurement TechniqueabstractThere are many paradigms available to address the unique and complex problems introduced with parallel programming. These complexities have implications for computer science education as ubiquitous multi-core computers drive the need for programmers to understand parallelism. One major obstacle to student learning of parallel programming is that there is very little human factors evidence comparing the different techniques to one another, so there is no clear direction on which techniques should be taught and how. We performed a randomized controlled trial using 88 university-level computer science student participants performing three identical tasks to examine the question of whether or not there are measurable differences in programming performance between two paradigms for concurrent programming: threads compared to process-oriented programming based on Communicating Sequential Processes. We measured both time on task and programming accuracy using an automated token accuracy map (TAM) technique. Our results showed trade-offs between the paradigms using both metrics and the TAMs provided further insight about specific areas of difficulty in comprehension. Patrick Daleiden, Andreas Stefik, Phillip Merlin Uesbeck, Jan Pedersen 0001 |
ACM Trans. Comput. Educ. | 2 |
| 2020 | Establishing Vibration-Based Tactile Line Profiles for Use in Multimodal GraphicsabstractVibration plays a significant role in the way users interact with touchscreens. For many users, vibration affords tactile alerts and other enhancements. For eyes-free users and users with visual impairments, vibration can also serve a more primary role in the user interface, such as indicating streets on maps, conveying information about graphs, or even specifying basic graphics. However, vibration is rarely used in current user interfaces beyond basic cuing. Furthermore, designers and developers who do actually use vibration more extensively are often unable to determine the exact properties of the vibration signals they are implementing, due to out-of-the-box software and hardware limitations. We make two contributions in this work. First, we investigate the contextual properties of touchscreen vibrations and how vibrations can be used to effectively convey traditional, embossed elements, such as dashes and dots. To do so, we developed an open source, Android-based library to generate vibrations that are perceptually salient and intuitive, improving upon existing vibration libraries. Second, we conducted a user study with 26 blind or visually impaired users to evaluate and categorize the effects with respect to traditional tactile line profiles. We have established a range of vibration effects that can be reliably generated by our haptic library and are perceptible and distinguishable by users. Jennifer L. Tennison, Phillip Merlin Uesbeck, Nicholas A. Giudice, Andreas Stefik, Derrick W. Smith, Jenna L. Gorlewicz |
ACM Trans. Appl. Percept. | 4 |
| 2019 | A Randomized Controlled Trial on the Wild Wild West of Scientific Computing with Student LearnersabstractScientific computing has become an area of growing importance. Across fields such as biology, education, physics, or others, people are increasingly using scientific computing to model and understand the world around them. Despite the clear need, almost no systematic analysis has been conducted on how students in fields outside of computer science learn to program in the context of scientific computing. Given that many fields do not explicitly teach much programming to their students, they may have to learn this important skill on their own. To help, using rigorous quantitative and qualitative methods, we looked at the process 154 students followed in the context of a randomized controlled trial on alternative styles of programming that can be used in R. Our results suggest that the barriers students face in scientific computing are non-trivial and this work has two core implications: 1) students learning scientific computing on their own struggle significantly in many different ways, even if they have had prior programming training, and 2) the design of the current generation of scientific computing feels like the wild-wild west and the designs can be improved in ways we will enumerate. Timothy Rafalski, Phillip Merlin Uesbeck, Cristina Panks-Meloney, Patrick Daleiden, William Allee, Amelia McNamara, Andreas Stefik |
ICER | 7 |
| 2019 | Access to Computing Education for Students with DisabilitiesabstractApproximately 10% of computer science and engineering majors have a disability. Students with disabilities are more likely to drop out of the major than those without disabilities. At the K-12 level, many tools used to teach computing have limited accessibility to students with disabilities. This BOF will bring together individuals who are interested in increasing the representation of students with disabilities in computing and improving their success. Participants will share strategies to help each other do a better job of including these students in our classes and research projects. Resources, including those produced by AccessComputing and AccessCSforAll, will be shared. Richard E. Ladner, Andreas Stefik, Amy J. Ko, Brianna Blaser |
SIGCSE | 2 |
| 2019 | Computer Science Principles for Teachers of Blind and Visually Impaired StudentsabstractThe College Board's AP Computer Science Principles (CSP) content has become a major new course for introducing K-12 students to the discipline. The course was designed for many reasons, but one major goal was to broaden participation. While significant work has been completed toward equity by many research groups, we know of no systematic analysis of CSP content created by major vendors in relation to accessibility for students with disabilities, especially those who are blind or visually impaired. In this experience report, we discuss two major actions by our team to make CSP more accessible. First, with the help of accessibility experts and teachers, we modified the entire Code.org CSP course to make it accessible. Second, we conducted a one-week professional development workshop in the summer of 2018 for teachers of blind or visually impaired students in order to help them prepare to teach CSP or support those who do. We report here on lessons learned that are useful to teachers who have blind or visually impaired students in their classes, to AP CSP curriculum providers, and to the College Board. Andreas Stefik, Richard E. Ladner, William Allee, Sean Mealin |
SIGCSE | 1 |
| 2017 | Access to Computing Education for Students with Disabilities (Abstract Only)abstractApproximately 10% of computer science and engineering majors have a disability of some kind. Students with disabilities are more likely to drop out of the major than those without disabilities. At the K-12 level, many tools used to teach computing are inaccessible to students with disabilities. This BOF will bring together individuals who are interested in increasing the representation of students with disabilities in computing and improving their success. Participants will share strategies and promising practices to help each other do a better job of including these students in our classes and research projects. Resources, including those produced by AccessComputing and AccessCSforAll (formally AccessCS10K) will be shared. Handouts and stickers will be given out. Richard E. Ladner, Andreas Stefik, Daniela Marghitu |
SIGCSE | 2 |
| 2017 | The Quorum Programming Language (Abstract Only)abstractQuorum is a relatively new programming language that was originally designed for students with disabilities. In recent years, as its adoption has increased worldwide in K-12 (largely in middle/high school) and at universities, it has expanded to be a powerful, commercial-grade, programming language that includes support for 3D gaming, music, and other fun and creative activities. While new features are designed for all, they maintain compatibility for people with disabilities, including a novel way for individuals who are blind to create 3D games. Finally, Quorum is the first language to use human-factors evidence from both field data and randomized controlled trials in its design. This approach provides the broader research community an organized way to influence the design of the language over time according to evidence based practices. We call this approach evidence-oriented programming. A laptop would help participants follow along with the session and handouts will be provided. Quorum can be found at https://www.quorumlanguage.com/. Andreas Stefik, Richard E. Ladner |
SIGCSE | 1 |
| 2016 | An empirical study on the impact of C++ lambdas and programmer experienceabstractLambda functions have become prevalent in mainstream programming languages, as they are increasingly introduced to widely used object oriented programming languages such as Java and C++. Some in the scientific literature argue that this feature increases programmer productivity, ease of reading, and makes parallel programming easier. Others are less convinced, citing concerns that the use of lambdas makes debugging harder. This thesis describes the design, execution and results of an experiment to test the impact of using lambda functions compared to the iterator design pattern for iteration tasks, as a first step in evaluating these claims. The approach is a randomized controlled trial, which focuses on the percentage of tasks completed, number of compiler errors, the percentage of time to fix such errors, and the amount of time it takes to complete programming tasks correctly. The overall goal is to investigate, if lambda functions have an impact on the ability of developers to complete tasks, or the amount of time taken to complete them. Additionally, it is tested if developers introduce more errors while using lambda functions and if fixing errors takes them more time. Lastly, the impact of experience level on productivity is evaluated by comparing the performance of participants from different levels of experience with one-another. Participants were assigned one of five levels based on their progress in the computer science major. The five levels were freshman, sophomore, junior, senior and professional. The professional level was assigned to individuals out of school with 5 or more years of professional experience. Results show that the impact of using lambdas, as opposed to iterators, on the number of tasks completed was significant. The impact on time to completion, number of errors introduced during the experiment and times spent fixing errors were also found to be significant. The analysis of the difference between the different levels of experience also shows a significant difference. The percentage of time spent on fixing compilation errors was 56.37% for the lambda group while it was 44.2% for the control group with 3.5% of the variance being explained by the group difference. 45.7% of the variance in the sample was explained by the difference between the level of education. Therefore, this study suggests that earlier findings that student results are comparable with the results of professional developers are to be considered carefully. Phillip Merlin Uesbeck, Andreas Stefik, Stefan Hanenberg, Jan Pedersen 0001, Patrick Daleiden |
ICSE | 2 |
| 2016 | Universal Access to Computing Education (Abstract Only)abstractApproximately 10% of computer science and engineering majors have a disability of some kind. Students with disabilities are more likely to drop out of the major than those without disabilities. This BOF will bring together individuals who are interested in increasing the representation of students with disabilities in computing and improving their success. Participants will share strategies and promising practices to help each other do a better job of including these students in our classes and research projects. Resources, including those produced by AccessComputing (uw.edu/accesscomputing) and AccessCS10K (uw.edu/accesscomputing/accesscs10k), will be shared. Richard E. Ladner, Brianna Blaser, Andreas Stefik, Daniela Marghitu |
SIGCSE | 3 |
| 2016 | Raising the Awareness of Accessibility Needs in Block Languages (Abstract Only)abstractBlock languages (e.g., Scratch, Snap!, Alice, App Inventor, Blockly) offer a gentle introduction to programming and have been adopted widely in both K-12 and CS0 courses. However, block languages often are dependent on the mouse/keyboard for input and typically are visual in their output and representation. Because of these dependencies, students with a disability (e.g., mobility limitations or vision impairment) generally are unable to use block languages, thereby reducing the opportunities for broader participation in computational learning activities. Given the increasing need to broaden the participation of computing to those with diverse skills and backgrounds, it is important that the tools used to initiate the earliest entre into computing do not erect immediate roadblocks that impede initial interest and opportunity. There are many variations of user interfaces and assistive technologies that benefit those who may have difficulties utilizing traditional Graphical User Interfaces (GUIs), but these tools often cannot be used universally across block languages. As more block languages are being developed and integrated into K12 and University curriculum, it is imperative that accessible solutions are discussed and implemented. These discussions require participation from the block language developer community, accessible computing community, and those educators who encounter accessibility needs among the students in their classrooms. The goal of this lightning talk is to call attention to the need for more accessible block-based programming environments and to spark conversation surrounding possible standard accessibility APIs that could possibly be supported by block language environment tool developers. Amber Wagner, Jeffrey G. Gray, Daniela Marghitu, Andreas Stefik |
SIGCSE | 4 |
| 2016 | Understanding Programming Expertise: An Empirical Study of Phasic Brain Wave ChangesabstractRecent decades have seen a resurgence of interest in electroencephalography (EEG), as neuroscience develops new models of cognition and refines old ones, associating them with detectable indicators of brain activity. This article presents a more direct measure of programmer expertise, derived from noninvasive observation of the brain’s electrical activity. This article provides a foundational approach for investigating the role of expertise in programming language comprehension, showing that this electrical activity in the brain can indicate (1) prior programming experience by class level (current state of progression through an undergraduate computer science program), and (2) self-reported experience levels. Igor Crk, Timothy Kluthe, Andreas Stefik |
ACM Trans. Comput. Hum. Interact. | 3 |
| 2015 | Introduction to AccessCS10K and Accessible Tools for Teaching ProgrammingabstractThis special session is for SIGCSE attendees who are interested in K-12 computing education, especially those who have or are planning to have a NSF CS 10K project. A major goal of CS 10K is broadening participation, which includes providing computing education to students with disabilities. This interactive session will introduce attendees to AccessCS10K, a project to help teachers include students with disabilities in their K-12 computing classes. Furthermore, attendees will have a hands-on experience with the Quorum language, a language designed for youth regardless of disability. It is expected that attendees to the special session will be better prepared to include a student with a disability in their computing classes or to help other teachers do the same. Andreas Stefik, Richard E. Ladner |
SIGCSE | 1 |
| 2015 | Comparing development approaches and reuse strategies: An empirical evaluation of developer views from the aerospace industry
Julia F. Varnell-Sarjeant, Anneliese Amschler Andrews, Joseph Lucente, Andreas Stefik |
Inf. Softw. Technol. | 4 |
| 2014 | How do API documentation and static typing affect API usability?abstractWhen developers use Application Programming Interfaces (APIs), they often rely on documentation to assist their tasks. In previous studies, we reported evidence indicating that static type systems acted as a form of implicit documentation, benefiting developer productivity. Such implicit documentation is easier to maintain, given it is enforced by the compiler, but previous experiments tested users without any explicit documentation. In this paper, we report on a controlled experiment and an exploratory study comparing the impact of using documentation and a static or dynamic type system on a development task. Results of our study both confirm previous findings and show that the benefits of static typing are strengthened with explicit documentation, but that this was not as strongly felt with dynamically typed languages. Stefan Endrikat, Stefan Hanenberg, Romain Robbes, Andreas Stefik |
ICSE | 4 |
| 2014 | What is the foundation of evidence of human factors decisions in language design? an empirical study on programming language workshopsabstractIn recent years, the programming language design community has engaged in rigorous debate on the role of empirical evidence in the design of general purpose programming languages. Some scholars contend that the language community has failed to embrace a form of evidence that is non-controversial in other disciplines (e.g., medicine, biology, psychology, sociology, physics, chemistry), while others argue that a science of language design is unrealistic. While the discussion will likely persist for some time, we begin here a systematic evaluation of the use of empirical evidence with human users, documenting, paper-by-paper, the evidence provided for human factors decisions, beginning with 359 papers from the workshops PPIG, Plateau, and ESP. This preliminary work provides the following contributions: an analysis of the 1) overall quantity and quality of empirical evidence used in the workshops, and of the 2) overall significant challenges to reliably coding academic papers. We hope that, once complete, this long-term research project will serve as a practical catalog designers can use when evaluating the impact of a language feature on human users. Andreas Stefik, Stefan Hanenberg, Mark McKenney, Anneliese Amschler Andrews, Srinivas Kalyan Yellanki, Susanna Kiwala |
ICPC | 1 |
| 2014 | An empirical study on the impact of static typing on software maintainability
Stefan Hanenberg, Sebastian Kleinschmager, Romain Robbes, Éric Tanter, Andreas Stefik |
Empir. Softw. Eng. | 5 |
| 2014 | Variation factors in the design and analysis of replicated controlled experiments - Three (dis)similar studies on inspections versus unit testing
Per Runeson, Andreas Stefik, Anneliese Amschler Andrews |
Empir. Softw. Eng. | 2 |
| 2013 | An Empirical Investigation into Programming Language SyntaxabstractRecent studies in the literature have shown that syntax remains a significant barrier to novice computer science students in the field. While this syntax barrier is known to exist, whether and how it varies across programming languages has not been carefully investigated. For this article, we conducted four empirical studies on programming language syntax as part of a larger analysis into the, so called, programming language wars. We first present two surveys conducted with students on the intuitiveness of syntax, which we used to garner formative clues on what words and symbols might be easy for novices to understand. We followed up with two studies on the accuracy rates of novices using a total of six programming languages: Ruby, Java, Perl, Python, Randomo, and Quorum. Randomo was designed by randomly choosing some keywords from the ASCII table (a metaphorical placebo). To our surprise, we found that languages using a more traditional C-style syntax (both Perl and Java) did not afford accuracy rates significantly higher than a language with randomly generated keywords, but that languages which deviate (Quorum, Python, and Ruby) did. These results, including the specifics of syntax that are particularly problematic for novices, may help teachers of introductory programming courses in choosing appropriate first languages and in helping students to overcome the challenges they face with syntax. Andreas Stefik, Susanna Kiwala |
ACM Trans. Comput. Educ. | 1 |
| 2012 | Do static type systems improve the maintainability of software systems? An empirical studyabstractStatic type systems play an essential role in contemporary programming languages. Despite their importance, whether static type systems influence human software development capabilities remains an open question. One frequently mentioned argument for static type systems is that they improve the maintainability of software systems - an often used claim for which there is no empirical evidence. This paper describes an experiment which tests whether static type systems improve the maintainability of software systems. The results show rigorous empirical evidence that static type are indeed beneficial to these activities, except for fixing semantic errors. Sebastian Kleinschmager, Stefan Hanenberg, Romain Robbes, Éric Tanter, Andreas Stefik |
ICPC | 5 |
| 2012 | An empirical study of the influence of static type systems on the usability of undocumented softwareabstractAbstract Although the study of static and dynamic type systems plays a major role in research, relatively little is known about the impact of type systems on software development. Perhaps one of the more common arguments for static type systems in languages such as Java or C++ is that they require developers to annotate their code with type names, which is thus claimed to improve the documentation of software. In contrast, one common argument against static type systems is that they decrease flexibility, which may make them harder to use. While these arguments are found in the literature, rigorous empirical evidence is lacking. We report on a controlled experiment where 27 subjects performed programming tasks on an undocumented API with a static type system (requiring type annotations) as well as a dynamic type system (which does not). Our results show that for some tasks, programmers had faster completion times using a static type system, while for others, the opposite held. We conduct an exploratory study to try and theorize why. Clemens Mayer, Stefan Hanenberg, Romain Robbes, Éric Tanter, Andreas Stefik |
OOPSLA | 5 |
| 2012 | Improving the accessibility of computing enrichment programs (abstract only)abstractMany wonderful enrichment programs have been created to introduce young people to computing, but with little attention to making them accessible to students with disabilities. In this workshop participants will learn from practitioners who have introduced computing and programming to young people with disabilities. They will also learn first-hand from students with disabilities about their needs in learning programming. There will be breakout sessions for participants to apply what they have learned to improve existing enrichment programs such as Alice, Arduino, Scratch, Kodu, App Inventor, Greenfoot, Lego Mindstorms, Processing, and Computer Science Unplugged. Richard E. Ladner, Karen Alkoby, Jeffrey P. Bigham, Stephanie Ludi, Daniela Marghitu, Andreas Stefik |
SIGCSE | 6 |
| 2011 | Toward Intuitive Programming LanguagesabstractModern text-based computer programming languages use syntax, semantics, and APIs to represent actions a computer will perform. Historically, the design of these languages has largely escaped the critical gaze of peer review, relying more on expert opinions than robust scientific methodologies. In this paper, we pose a question - is it possible to create a programming language where the syntax, semantics, and API design is based upon rigorous data collection and the scientific method? We have undertaken a long-term project to develop a computer programming language, called Hop, where each language decision is based upon empirical metrics gathered from human studies. While such a design may not universally benefit all programmers, our hope is that such a procedure may make our community's language design decisions more objective and transparent. Andreas Stefik, Susanna Kiwala, Kim Slattery, Melissa Stefik |
ICPC | 1 |
| 2011 | On the design of an educational infrastructure for the blind and visually impaired in computer scienceabstractThe blind and visually impaired community is significantly underrepresented in computer science. Students who wish to enter the discipline must overcome significant technological and educational barriers to succeed. In an attempt to help this population, we are engaged in a three-year research project to build an educational infrastructure for blind and visually impaired middle and high school students. Our primary research goal is to begin forging a multi-sensory educational infrastructure for the blind across the United States. We present here two preliminary results from this research: 1) a new auditory programming environment called Sodbeans, a programming language called Hop, and a multi-sensory (sound and touch) curriculum, and 2) an empirical study of our first summer workshop with the blind students. Results show that students reported a significant increase in programming self-efficacy after participating in our camp. Andreas Stefik, Christopher D. Hundhausen, Derrick W. Smith |
SIGCSE | 1 |
| 2011 | An empirical investigation into the design of auditory cues to enhance computer program comprehension
Andreas Stefik, Christopher D. Hundhausen, Robert Patterson |
Int. J. Hum. Comput. Stud. | 1 |
| 2011 | Empirical studies on programming language stimuli
Andreas Stefik, Ed Gellenbeck |
Softw. Qual. J. | 1 |
| 2010 | Testing coupling relationships in object-oriented programsabstractAbstract As we move toward developing object‐oriented (OO) programs, the complexity traditionally found in functions and procedures is moving to the connections among components. Different faults occur when components are integrated to form higher‐level structures that aggregate the behavior and state. Consequently, we need to place more effort on testing the connections among components. Although OO technologies provide abstraction mechanisms for building components that can then be integrated to form applications, it also adds new compositional relations that can contain faults. This paper describes techniques for analyzing and testing the polymorphic relationships that occur in OO software. The techniques adapt traditional data flow coverage criteria to consider definitions and uses among state variables of classes, particularly in the presence of inheritance, dynamic binding, and polymorphic overriding of state variables and methods. The application of these techniques can result in an increased ability to find faults and to create an overall higher quality software. Copyright © 2010 John Wiley & Sons, Ltd. Roger T. Alexander, A. Jefferson Offutt, Andreas Stefik |
Softw. Test. Verification Reliab. | 3 |
| 2009 | Evaluating prosodic cues as a means to disambiguate algebraic expressions: an empirical studyabstractThe automatic translation of written mathematical expressions to their spoken equivalent is a difficult task. Written mathematics makes use of specialized symbols and a 2-dimensional layout that is hard to translate into clear and unambiguous spoken words. Our approach is to use prosody to help listeners follow along to mathematical expressions spoken aloud with text-to-speech synthesized voices. To achieve this, we developed and empirically tested XSL transformation rules that automatically translate mathematical expressions marked-up with Presentation MathML into corresponding markup using the Speech Synthesis Markup Language (SSML). In this paper, we report on the results from an empirical study we conducted that showed that the simple insertion of pauses inside spoken mathematical expressions dramatically improved subjects' ability to disambiguate between two similar algebraic expressions. Result from our study should benefit designers of screen readers and related audio-based tools that produce spoken renderings of mathematical expressions. Ed Gellenbeck, Andreas Stefik |
ASSETS | 2 |
| 2009 | Using spoken text to aid debugging: An empirical studyabstractComprehending and debugging computer programs are inherently difficult tasks. The current approach to building program execution and debugging environments is to use exclusively visual stimuli. We present an alternative: the Sonified Omniscient Debugger (SOD), a program execution and debugging environment designed to output carefully chosen spoken auditory cues to supplement visual stimuli. Originally designed for the blind, earlier work suggested that SOD may benefit sighted programmers as well. We evaluate the SOD environment in a formal debugging experiment comparing 1) a visual debugger, 2) an auditory debugger, and 3) a multimedia debugger, which includes both the visual and auditory stimuli. Our results indicate that while auditory debuggers on their own are significantly less effective for sighted users when compared with visual and multimedia debuggers, multimedia debuggers might benefit sighted programmers under certain circumstances. Specifically, we found that while multimedia debuggers do not provide instant usability, once programmers have some practice, their performance improves under certain metrics. Andreas Stefik, Ed Gellenbeck |
ICPC | 1 |
| 2009 | SODBeansabstractComprehending and debugging computer programs are inherently difficult tasks for sighted programmers. These tasks are even more difficult for non-sighted programmers, who rely on audio-based representations of programs. The state-of-the-art approach to building program execution and debugging environments for non-sighted programmers is to retrofit existing visual environments with screen readers. Because of intrinsic differences in the ways humans process auditory and visual information, we argue that these environments are insensitive to the needs of the blind community. We present an alternative: SODBeans (the SonifiedOmniscient Debugger in Netbeans), a compiler, debugger, and accessibility architecture for Netbeans 6.5. Andreas Stefik, Andrew Haywood, Shahzada Mansoor, Brock Dunda, Dan Garcia 0001 |
ICPC | 1 |
| 2007 | WAD: A Feasibility study using the Wicked Audio DebuggerabstractContemporary programmers have a predominately visual experience. Computer code is read from sophisticated text editors, analyzed using visual tools, designed using UML, and debugged using a watch window. Little research has attempted to create tools for the non-sighted programmer, using either haptic or aural feedback. In this paper, we present WAD, the wicked audio debugger, a new debugger for Microsoft Visual Studio 2005 that sonifies computer code as an aid to the programmer. This paper has two primary contributions, namely the tool itself and the results of a feasibility study. We conducted this feasibility study to test participants' ability to comprehend dynamic program behavior, including tracking the value of state variables and control flow. Results of our feasibility study show that participants were able to comprehend approximately 86% of dynamic program behavior using audio only. Andreas Stefik, Roger T. Alexander, Robert Patterson, Jonathan Brown |
ICPC | 1 |
| 2006 | Layered Program Auralization: Using Music to Increase Runtime Program Comprehension and Debugging EffectivenessabstractLayered program auralization uses music for debugging computer code. Layers of musical structure represent the state and behavior of a computer program while it is running, taking advantage of metaphorical relationships between musical structure and programming constructs. This paper makes three contributions to the debugging and program auralization literature. We use cadences, recognizable patterns of chords that usually dictate an ending, to metaphorically represent nesting depth and hierarchical control structures. Auralizations, in our system, use more than one perceptual mapping at a time. Further, we decompose our auralizations into units, which we call layers. This decomposition is similar to using classes in object oriented programming. We see debugging as a time consuming, difficult task, and present a solution where music is played to the programmer during program execution. Our goal is to increase debugging effectiveness and to improve a programmer's comprehension of the runtime behavior of computer programs Andreas Stefik, Kelly Fitz, Roger T. Alexander |
ICPC | 1 |