Merge conflicts are an inevitable, but painful part of collaborative software development. Merge conflict resolution is nontrivial because it requires gathering information from disparate sources (e.g., a codebase, diffs between versions, history of changes, documentation etc.) and then piecing together not only this information, but the rationale and context behind the conflicting changes. Current tools offer inadequate support to developers trying to understand the context and impact of the conflicting changes. They also offer little support in evaluating potential resolutions. To improve conflict resolution tools, we first need to understand the information needs of conflict resolution and its underlying sensemaking process. In this paper, through in-situ observations of 10 conflict resolutions, we qualitatively investigate how developers collect and sensemake different conflict-related information and how they reach a resolution. We identified eight behaviour patterns that developers use. These patterns have implications for tool builders and developers alike. We identify specific areas that need improved tool support. We also highlight the sensemaking traps practitioners may fall into.

Open design discussion is a primary mechanism through which open source projects debate, make and document design decisions. However, there are open questions regarding how design discussions are conducted and what effect they have on the design quality of projects. Recent work has begun to investigate design discussions, but has thus far focused on a single communication channel, whereas many projects use multiple channels. In this study, we examine 37 Apache projects and their design discussions, the project’s design quality evolution, and the relationship between design discussion and design quality. A mixed method empirical analysis (data mining and a survey of 130 developers) shows that: I) 89.51% of all design discussions occur in project mailing list, II) both core and non-core developers participate in design discussions, but core developers implement more design related changes (67.06%), and III) the correlation between design discussions and design quality is small. We conclude the paper with several observations that form the foundation for future research and development.

Block-based programming languages are used by millions of people around the world. Blockly is a popular JavaScript library for creating visual block programming editors. To input a block, users employ a drag-and-drop input style. However, there are some limitations to this input style. We introduce a custom soft keyboard to input Blockly programs. This keyboard allows inputting, changing or editing blocks with a single touch. We evaluated the keyboard users' speed, number of touches, and errors while inputting a Blockly program and compared its performance with the drag-and-drop method. Our keyboard reduces the input errors by 68.37% and the keystrokes by 47.97 %. Moreover, it increases the input speed by 71.26% when compared to the drag-and-drop. The keyboard users perceived it to be physically less demanding with less effort than the drag-and-drop method. Moreover, participants rated the drag-and-drop method to have a higher frustration level. The Blockly keyboard was the preferred input method.

Background: Merge conflicts are a common occurrence in software development. Researchers have shown the negative impact of conflicts on the resulting code quality and the development workflow. Thus far, no one has investigated the effect of bad design (code smells) on merge conflicts. Aims: We posit that entities that exhibit certain types of code smells are more likely to be involved in a merge conflict. We also postulate that code elements that are both "smelly" and involved in a merge conflict are associated with other undesirable effects (more likely to be buggy). Method: We mined 143 repositories from GitHub and recreated 6,979 merge conflicts to obtain metrics about code changes and conflicts. We categorized conflicts into semantic or non-semantic, based on whether changes affected the Abstract Syntax Tree. For each conflicting change, we calculate the number of code smells and the number of future bug-fixes associated with the affected lines of code. Results: We found that entities that are smelly are three times more likely to be involved in merge conflicts. Method-level code smells (Blob Operation and Internal Duplication) are highly correlated with semantic conflicts. We also found that code that is smelly and experiences merge conflicts is more likely to be buggy. Conclusion: Bad code design not only impacts maintainability, it also impacts the day to day operations of a project, such as merging contributions, and negatively impacts the quality of the resulting code. Our findings indicate that research is needed to identify better ways to support merge conflict resolution to minimize its effect on code quality.

Fault masking happens when the effect of one fault serves to mask that of another fault for particular test inputs. The coupling effect is relied upon by testing practitioners to ensure that fault masking is rare. It states that complex faults are coupled to simple faults in such a way that a test data set that detects all simple faults in a program will detect a high percentage of the complex faults. While this effect has been empirically evaluated, our theoretical understanding of the coupling effect is as yet incomplete. Wah proposed a theory of the coupling effect on finite bijective (or near bijective) functions with the same domain and co-domain and assuming a uniform distribution for candidate functions. This model, however, was criticized as being too simple to model real systems, as it did not account for differing domain and co-domain in real programs, or for the syntactic neighborhood. We propose a new theory of fault coupling for general functions (with certain constraints). We show that there are two kinds of fault interactions, of which only the weak interaction can be modeled by the theory of the coupling effect. The strong interaction can produce faults that are semantically different from the original faults. These faults should hence be considered as independent atomic faults. Our analysis shows that the theory holds even when the effect of the syntactic neighborhood of the program is considered. We analyze numerous real-world programs with real faults to validate our hypothesis.

Free/Open Source Software developers come from a myriad of different backgrounds, and are driven to contribute to projects for a variety of different reasons, including compensation from corporations or foundations. Motivation can have a dramatic impact on how and what contribution an individual makes, as well as how tenacious they are. These contributions may align with the needs of the developer, the community, the organization funding the developer, or all of the above. Understanding how corporate sponsorship affects the social dynamics and evolution of Free/Open Source code and community is critical to fostering healthy communities. We present a case study of corporations contributing to the Linux Kernel. We find that corporate contributors contribute more code, but are less likely to participate in non-coding activities. This knowledge will help project leaders to better understand the dynamics of sponsorship, and help to steer resources.

The syntax-directed keyboard extension presented by Almusaly et al. in 2015 allows programmers to input Java source code with fewer errors and keystrokes compared to the soft QWERTY keyboard and it supports a comparable typing speed. While these results were obtained after only 10 minutes of practice, it is unclear how long term use affects performance. In this paper, we present an updated design for the original syntax-directed keyboard extension, replicate the original results, and evaluate the evolved design with Java programmers over eight sessions in a period of two weeks. Our results indicate that a programmer using the new keyboard extension for two weeks can input Java programs 16.5% faster (words per minute) than an expert QWERTY keyboard typist. In addition, we demonstrate that the efficiency and accuracy for inputting Java source code improves with repeated use over time and that perceived mental, physical, and temporal demands of the keyboard extension decrease over time.

Mutation analysis is a well known yet unfortunately costly method for measuring test suite quality. Researchers have proposed numerous mutation reduction strategies in order to reduce the high cost of mutation analysis, while preserving the representativeness of the original set of mutants. As mutation reduction is an area of active research, it is important to understand the limits of possible improvements. We theoretically and empirically investigate the limits of improvement in effectiveness from using mutation reduction strategies compared to random sampling. Using real-world open source programs as subjects, we find an absolute limit in improvement of effectiveness over random sampling- 13.078%. Given our findings with respect to absolute limits, one may ask: How effective are the extant mutation reduction strategies? We evaluate the effectiveness of multiple mutation reduction strategies in comparison to random sampling. We find that none of the mutation reduction strategies evaluated-many forms of operator selection, and stratified sampling (on operators or program elements)-produced an effectiveness advantage larger than 5% in comparison with random sampling. Given the poor performance of mutation selection strategies-they may have a negligible advantage at best, and often perform worse than random sampling- we caution practicing testers against applying mutation reduction strategies without adequate justification.

Pair programming has been shown to be a beneficial and popular technique for engaging students and improving learning outcomes in programming and related classes. While using pair programming in a collocated classroom setting is relatively straightforward, there is a strong lack of good tools and options for distributed classroom settings. The growth of such classes, from small structured seminars to massive open online courses (MOOCs), and the isolation and impoverished learning experience some students report in these, points to an urgent need for tools that support remote pair programming in a distributed educational setting. This paper explores the requirements and needs of online learners in Computer Science through a literature survey. To validate these requirements, we implement a collaborative development environment aimed at improving the learning experience through pair programming and a tights code-artifact feedback loop. We conclude by presenting the results of a user study in both collocated and distributed classroom settings.

Although mutation analysis is considered the best way to evaluate the effectiveness of a test suite, hefty computational cost often limits its use. To address this problem, various mutation reduction strategies have been proposed, all seeking to reduce the number of mutants while maintaining the representativeness of an exhaustive mutation analysis. While research has focused on the reduction achieved, the effectiveness of these strategies in selecting representative mutants, and the limits in doing so have not been investigated, either theoretically or empirically.

There are many papers about projects that release their software as open source. Our panel looks at why open source is a good idea for education, and how it can best be leveraged. The panel represents industry and academia and its members have years of experience in the field. Specific questions that panelists will address, in addition to their position statements, are:

Among the major questions that a practicing tester faces are deciding where to focus additional testing effort, and deciding when to stop testing. Test the least-tested code, and stop when all code is well-tested, is a reasonable answer. Many measures of "testedness" have been proposed; unfortunately, we do not know whether these are truly effective. In this paper we propose a novel evaluation of two of the most important and widely-used measures of test suite quality. The first measure is statement coverage, the simplest and best-known code coverage measure. The second measure is mutation score, a supposedly more powerful, though expensive, measure.

Context: Software decay is a key concern for large, long-lived software projects. Systems degrade over time as design and implementation compromises and exceptions pile up. Goal: Quantify design decay and understand how software projects deal with this issue. Method: We conducted an empirical study on the presence and evolution of code smells, used as an indicator of design degradation in 220 open source projects. Results: The best approach to maintain the quality of a project is to spend time reducing both software defects (bugs) and design issues (refactoring). We found that design issues are frequently ignored in favor of fixing defects. We also found that design issues have a higher chance of being fixed in the early stages of a project, and that efforts to correct these stall as projects mature and the code base grows, leading to a build-up of problems. Conclusions: From studying a large set of open source projects, our research suggests that while core contributors tend to fix design issues more often than non-core contributors, there is no difference once the relative quantity of commits is accounted for. We also show that design issues tend to build up over time.

Mutation analysis is considered the best method for measuring the adequacy of test suites. However, the number of test runs required for a full mutation analysis grows faster than project size, which is not feasible for real-world software projects, which often have more than a million lines of code. It is for projects of this size, however, that developers most need a method for evaluating the efficacy of a test suite. Various strategies have been proposed to deal with the explosion of mutants. However, these strategies at best reduce the number of mutants required to a fraction of overall mutants, which still grows with program size. Running, e.g., 5% of all mutants of a 2MLOC program usually requires analyzing over 100,000 mutants. Similarly, while various approaches have been proposed to tackle equivalent mutants, none completely eliminate the problem, and the fraction of equivalent mutants remaining is hard to estimate, often requiring manual analysis of equivalence. In this paper, we provide both theoretical analysis and empirical evidence that a small constant sample of mutants yields statistically similar results to running a full mutation analysis, regardless of the size of the program or similarity between mutants. We show that a similar approach, using a constant sample of inputs can estimate the degree of stubbornness in mutants remaining to a high degree of statistical confidence, and provide a mutation analysis framework for Python that incorporates the analysis of stubbornness of mutants.

Formal verification has advanced to the point that developers can verify the correctness of small, critical modules. Unfortunately, despite considerable efforts, determining if a "verification" verifies what the author intends is still difficult. Previous approaches are difficult to understand and often limited in applicability. Developers need verification coverage in terms of the software they are verifying, not model checking diagnostics. We propose a methodology to allow developers to determine (and correct) what it is that they have verified, and tools to support that methodology. Our basic approach is based on a novel variation of mutation analysis and the idea of verification driven by falsification. We use the CBMC model checker to show that this approach is applicable not only to simple data structures and sorting routines, and verification of a routine in Mozilla's JavaScript engine, but to understanding an ongoing effort to verify the Linux kernel Read-Copy-Update (RCU) mechanism.

One of the key challenges of developers testing code is determining a test suite's quality -- its ability to find faults. The most common approach is to use code coverage as a measure for test suite quality, and diminishing returns in coverage or high absolute coverage as a stopping rule. In testing research, suite quality is often evaluated by a suite's ability to kill mutants (artificially seeded potential faults). Determining which criteria best predict mutation kills is critical to practical estimation of test suite quality. Previous work has only used small sets of programs, and usually compares multiple suites for a single program. Practitioners, however, seldom compare suites --- they evaluate one suite. Using suites (both manual and automatically generated) from a large set of real-world open-source projects shows that evaluation results differ from those for suite-comparison: statement (not block, branch, or path) coverage predicts mutation kills best.

Mutation analysis is often used to compare the effectiveness of different test suites or testing techniques. One of the main assumptions underlying this technique is the Competent Programmer Hypothesis, which proposes that programs are very close to a correct version, or that the difference between current and correct code for each fault is very small. Researchers have assumed on the basis of the Competent Programmer Hypothesis that the faults produced by mutation analysis are similar to real faults. While there exists some evidence that supports this assumption, these studies are based on analysis of a limited and potentially non-representative set of programs and are hence not conclusive. In this paper, we separately investigate the characteristics of bug-fixes and other changes in a very large set of randomly selected projects using four different programming languages. Our analysis suggests that a typical fault involves about three to four tokens, and is seldom equivalent to any traditional mutation operator. We also find the most frequently occurring syntactical patterns, and identify the factors that affect the real bug-fix change distribution. Our analysis suggests that different languages have different distributions, which in turn suggests that operators optimal in one language may not be optimal for others. Moreover, our results suggest that mutation analysis stands in need of better empirical support of the connection between mutant detection and detection of actual program faults in a larger body of real programs.

Researchers have investigated the lack of diversity in Free/Open Source Software (FOSS) communities, but there have been few studies on age diversity. We interviewed 11 older FOSS contributors and 6 FOSS community leaders (of any age). This formative study reports on 4 key findings from those interviews: 1) motivations of older contributors, 2) benefits and challenges to contribution, 3) older adults' views on discrimination in FOSS, and 4) ways in which older adults enrich FOSS communities. We found that older adults' contributions are driven by intrinsic motivation, altruism, and community identification. In older adults' most recent contributions, we found that there were more social than technical challenges to participation. Interestingly, the majority of older adults claimed to have witnessed discrimination towards others in FOSS, especially against non-native English speakers and women. This stands in contrast to what the general male FOSS developer population reports. Participants identified 10 ways that older adults add value to FOSS communities. We conclude with guidelines for onboarding older adults.

Free/Open Source Software projects often rely on users submitting bug reports. However, reports submitted by novice users may lack information critical to developers, and the process may be intimidating and difficult. To gather more and better data, projects deploy automatic crash reporting tools, which capture stack traces and memory dumps when a crash occurs. These systems potentially generate large volumes of data, which may overwhelm developers, and their presence may discourage users from submitting traditional bug reports. In this paper, we examine Mozilla's automatic crash reporting system and how it affects their bug triaging process. We find that fewer than 0.00009% of crash reports end up in a bug report, but as many as 2.33% of bug reports have data from crash reports added. Feedback from developers shows that despite some problems, these systems are valuable. We conclude with a discussion of the pros and cons of automatic crash reporting systems.

The global population is aging rapidly, and older adults are becoming increasingly technically savvy. This paper explores ways to engage these individuals to contribute to free/open source software (FOSS) projects. We conducted a pilot diary study to explore motivations, barriers, and the contribution processes of first-time contributors in a real time, qualitative manner. In addition, we measured their self-efficacy before and after their participation. We found that what drove participants were intrinsic motivations, altruism, and internal values, which differed from previous work with older adults and with the general FOSS population. We also found that self-efficacy did not change significantly, even when participants encountered significant barriers or setbacks. The top 3 barriers were lack of communication, installation issues, and documentation issues. We found that asking for and receiving help, and avoiding difficult development environments were more likely to lead to success. To verify these results, we encourage a future large-scale diary study that involves multiple demographics. Given our pilot study, we recommend that future outreach efforts involving older adults focus on how to effectively communicate and build community amongst older contributors.

Older adults are increasingly savvy consumers of smartphone-based health solutions and information. These technologies may enable older adults to age-in-place more successfully. However, many app creators fail to do needs assessments of their end-users. To rectify this issue, we involved older adults (aged 65+) in the beginning stages of designing a mobile health and wellness application. We conducted a participatory design study, where 5 groups of older adults created 5 designs. Four groups identified at least 1 health metric not currently offered in either the iPhone app store or the Google Play store. At the end of the sessions we administered a questionnaire to determine what health topics participants would like to track via smartphone or tablet. The designs included 13 health topics that were not on the questionnaire. Seventeen of eighteen participants expressed interest in tracking health metrics using a smartphone/tablet despite having little experience with these devices. This shows that older adults have unique ideas that are not being considered by current technology designers. We conclude with recommendations for future development, and propose continuing to involve to older adults in participatory design.

Researchers often use participatory design -- involving endusers in technology ideation -- as this is found to lead to more useful and relevant products. Researchers have sought to involve older adults in the design of emerging technologies like smartphones, with which older adults often have little experience. Therefore, their effectiveness as co-designers could be questioned. We examine whether older adults can create novel design ideas, and whether critiquing existing applications prior to ideation helps or hinders creativity. Panelists from industry and academia evaluated design ideas generated by focus groups of older adults. Out of five groups, the most creative design idea came from one with no smartphone experience or critique exposure. We found that while only some designs scored high on the novelty dimension of creativity, participants were enthusiastic about participating and adapted quickly. We found evidence that critiquing existing applications prior to ideation did more harm than good, potentially due to design fixation. We recommend continuing to involve older adults in the technology design ideation phase.

There is a growing interest in bringing online and streaming content to the television. Gaming platforms such as the PS3, Xbox 360 and Wii are at the center of this digital convergence; platforms for accessing new media services. This presents a number of interface challenges, as controllers designed for gaming have to be adapted to accessing online content. This paper presents a user study examining the limitations and affordances of novel game controllers in an interactive TV (iTV) context and compares them to "second display" approaches using tablets. We look at task completion times, accuracy and user satisfaction across a number of tasks and find that the Wiimote is most liked and performed best in almost all tasks. Participants found the Kinect difficult to use, which led to slow performance and high error rates. We discuss challenges and opportunities for the future convergence of game consoles and iTV.

Tools aimed at helping users safely navigate the web and safeguard themselves against potential online predators have become reasonably common. Currently there are two families of tools; heuristics analysis tools that test websites directly using automated scripts and programs, and community based tools where users rate websites and write reviews for the benefit of others. In this paper we examine the relative strengths and weaknesses of each technique, whether these techniques are compatible, and how community feedback can be combined with heuristic-based evaluations. In order to do this we conduct a large-scale comparison of the ratings of heuristic and community based tools, and explore novel methods for abstracting key information from user comments, which could be used to add context and nuance to heuristic based ratings. We find that heuristic and community based ratings are highly complementary, and can be combined to potentially guide users to make more informed decisions.

Free/Open Source Software (FOSS) communities often use open bug reporting to allow users to participate by reporting bugs. This practice can lead to more duplicate reports, as users can be less rigorous about researching existing bug reports. This paper examines how FOSS projects deal with duplicate bug reports. We examined 12 FOSS projects: 4 small, 4 medium and 4 large, where size was determined by number of code contributors. First, we found that contrary to what has been reported from studies of individual large projects like Mozilla and Eclipse, duplicate bug reports are a problem for FOSS projects, especially medium-sized, which struggle with a large number of submissions without the resources of large projects. Second, we found that the focus of a project does not affect the number of duplicate bug reports. Our findings indicate a need for additional scaffolding and training for bug reporters.

In the field of Human-Computer Interaction, provenance refers to the history and genealogy of a document or file. Provenance helps us to understand the evolution and relationships of files; how and when different versions of a document were created, or how different documents in a collection build on each other through copy-paste events. Though methods for tracking provenance and the subsequent use of this meta-data have been proposed and developed into tools, there have been no studies documenting the types and frequency of provenance events in typical computer use. This is knowledge essential for the design of efficient query methods and information displays. We conducted a longitudinal study of knowledge workers at Intel Corporation tracking provenance events in their computer use. We also interviewed knowledge workers to determine the effectiveness of provenance cues for document recall. Our data shows that provenance relationships are common, and provenance cues aid recall.

In co-located software development, diagramming practices, such as sketching ideas out with a pen and paper, support the creative process and allow designers to shape, analyze, and communicate their ideas. This study focuses on the diagramming practices used in the design of Open Source Software (OSS), where the norm is highly distributed group work. In OSS, text-based communication (e.g., mailing lists) dominates, and sketching and drawing diagrams collaboratively remains difficult due to the barriers imposed by distance and technology. Previous studies have examined these practices and barriers in the context of individual projects. To understand how contributors across OSS projects use diagrams in design-related activities, we conducted a survey of 230 contributors from 40 different OSS projects, and interviewed eight participants. Our results show that although contributors understand the advantages of using diagrams for design-related activities, diagrams are infrequently used in OSS. This motivated us to examine how and why diagramming occurs, and the factors that prevent widespread use in OSS. Finally, we propose new ideas for supporting design activities in OSS projects.

Some of the most interesting differences between Open Source Software (OSS) development and commercial co-located software development lie in the communication and collaboration practices of these two groups of developers. One interesting practice is that of diagramming. Though well studied and important in many aspects of co-located software development (including communication and collaboration among developers), its role in OSS development has not been thoroughly studied. In this paper, we report our investigation on how and why Ubuntu contributors use diagrams in their work. Our study shows that diagrams are not actively used in many scenarios where they commonly would in co-located software development efforts. We describe differences in the use and practices of diagramming, their possible reasons, and present design considerations for potential systems aimed at better supporting diagram use in OSS development.

In this paper we introduce the iWatch web crawler, a tool designed to catalogue and analyze online data practices and the use of privacy related indicators and technologies. Our goal in developing iWatch was to make possible a new type of analysis of trends, the impact of legislation on practices, and geographic and social differences online. In this paper we present preliminary findings from two sets of data collected 15 months apart and analyzed with this tool. Our combined samples included more than 240,000 pages from over 24,000 domains and 47 different countries. In addition to providing useful and needed data on the state of online data practices, we show that iWatch is a promising approach to the study of the web ecosystem.

Studies have repeatedly shown that users are increasingly concerned about their privacy when they go online. In response to both public interest and regulatory pressures, privacy policies have become almost ubiquitous. An estimated 77% of websites now post a privacy policy. These policies differ greatly from site to site, and often address issues that are different from those that users care about. They are in most cases the users' only source of information.This paper evaluates the usability of online privacy policies, as well as the practice of posting them. We analyze 64 current privacy policies, their accessibility, writing, content and evolution over time. We examine how well these policies meet user needs and how they can be improved. We determine that significant changes need to be made to current practice to meet regulatory and usability requirements.

In this paper, we examine what types of reputation information users find valuable when selecting someone to interact with in online environments. In an online experiment, we asked users to imagine that they were looking for a partner for a social chat. We found that similarity to the user and ratings from the user's friends were the most valuable pieces of reputation information when selecting chat partners. The context in which reputations were used (social chat, game or newsgroup) affected the self-reported utility of the pieces of reputation information

One of the most robust findings in the sociological literature is the positive effect of communication on cooperation and trust. When individuals are able to communicate, cooperation increases significantly. How does the choice of communication modality influence this effect? We adapt the social dilemma research paradigm to quantitatively analyze different modes of communication. Using this method, we compare four forms of communication: no communication, text-chat, text-to-speech, and voice. We found statistically significant differences between different forms of communication, with the voice condition resulting in the highest levels of cooperation. Our results highlight the importance of striving towards the use of more immediate forms of communication in online environments, especially where trust and cooperation are essential. In addition, our research demonstrates the applicability of the social dilemma paradigm in testing the extent to which communication modalities promote the development of trust and cooperation.