When developers run tests after making code changes, they may encounter test failures from flaky tests, which are tests that can non-deterministically pass or fail on the same version of code. Prior work has found “timing dependence” to be a top cause of this non-determinism, i.e., tests may pass or fail depending on the timing of asynchronous callbacks or different thread interleavings that can occur when thread executions run faster or slower relative to others. Similar to how one debugs and fixes normal test failures, developers need to be able to reliably reproduce flaky-test failures. However, many of these failures can be extremely unlikely to occur (e.g., failing only once out of 10,000 runs in prior work), making it costly for developers to reproduce the failures. We present FlakeRake, an automated approach for reproducing timing-dependent (TD) flaky-test failures by inserting well-placed sleep calls, which temporarily pauses one thread or task and allows another to overtake it. When applied to an existing dataset of known flaky-test failures, FlakeRake is able to reproduce the exact same failure at least once for 136 failures, whereas simply rerunning each test 10,000 times reproduces only 115 failures or rerunning the entire test suites 10,000 times reproduces only 127 failures. For each failure that can be reproduced, we find that FlakeRake can reliably reproduce (>50% of the time) 107 failures, while rerunning just the flaky test or the entire test suite could not reliably reproduce any failure. We also find that if a developer needs to reproduce a failure six or more times, using FlakeRake (including the one-time cost to search for sleep calls) takes less time to reproduce that many failures than continually rerunning just the flaky test. Lastly, we inspect the sleep locations that FlakeRake outputs and provide insights for how one should cope with TD flaky tests.

Compliance reviews within a software organization are internal attempts to verify regulatory and security requirements during product development before its release. However, these reviews are not enough to adequately assess and address regulatory and security requirements throughout a software's development lifecycle. We believe requirements engineers can benefit from an improved understanding of how software practitioners treat and perceive compliance requirements. This paper describes an interview study seeking to understand how regulatory and security standard requirements are addressed, how burdensome they may be for businesses, and how our participants perceived them in the software development lifecycle. We interviewed 15 software practitioners from 13 organizations with different roles in the software development process and working in various industry domains, including big tech, healthcare, data analysis, finance, and small businesses. Our findings suggest that, for our participants, the software release process is the ultimate focus for regulatory and security compliance reviews. Also, most participants suggested that having a defined process for addressing compliance requirements was freeing rather than burdensome. Finally, participants generally saw compliance requirements as an investment for both employees and customers. These findings may be unintuitive, and we discuss seven lessons this work may hold for requirements engineering.

When working in large and complex codebases, developers face challenges using Find Usages to understand how to reuse classes and methods. To better understand these challenges, we conducted a small exploratory study with 4 participants. We found that developers often wasted time reading long lists of similar usages or prematurely focused on a single usage. Based on these findings, we hypothesized that clustering usages by the similarity of their surrounding context might enable developers to more rapidly understand how to use a function. To explore this idea, we designed and implemented Find Unique Usages, which extracts usages, computes a diff between pairs of usages, generates similarity scores, and uses these scores to form usage clusters. To evaluate this approach, we conducted a controlled experiment with 12 participants. We found that developers with Find Unique Usages were significantly faster, completing their task in 35% less time.