While symbolic execution (SE) can discover software vulnerabilities, it has received limited practical adoption. A key barrier is that SE requires human expertise to understand the program’s state and prioritize paths to analyze. Traditionally, users controlled SE through programmatic API calls, but recent tooling now implements graphical user interfaces (GUI). However, it is unclear how these new features affect human-SE performance. To understand this impact, we conducted a controlled experiment where 24 vulnerability discovery experts were tasked with analyzing a binary using an SE tool with either API or GUI-based features. From this study, we identify (1) experts’ SE process, and (2) the impact of GUI-based features on human-SE performance. Then we propose recommendations to improve SE tool design.

Internet-connected medical devices introduce complex cybersecurity risks that challenge the established practice of informed consent. It remains unclear how patients weigh these abstract, dynamic threats against concrete clinical benefits. We present findings from a large-scale (N=2,666) vignette-based experiment designed to uncover the factors driving patient decision-making. Participants chose whether to adopt a connected pacemaker, weighing its enhanced clinical outcomes against potential vulnerabilities. We systematically varied communication factors, including the source of risk information (e.g., clinician, FDA), risk framing, and the details of a subsequent vulnerability disclosure. Our results reveal patient choice hinges on pre-existing physician trust and risk framing. We did not observe any effect from the information’s source. We also find initial choices act as powerful anchors, and that detailed disclosures increase security confidence. Our work provides crucial empirical evidence on this trade-off, offering actionable guidance to better support informed consent for life-critical connected technologies.

Compared with preceding consumer technologies, virtual reality (VR) requires extensive collection of sensitive biometric data. On the Meta Quest 2 (the most popular consumer headset), application access to sensitive data is mediated by a permissions system adapted from Android’s model. In this work, we examine how VR’s immersive nature necessitates new permissions considerations beyond preceding technology and current VR devices. We analyze the Meta Quest 2’s permissions system, identifying where it diverges from Android's user-facing behavior surrounding biometric data. We then investigate the implications of both conventional Android and novel Quest permission flows through an in-person VR interview study with 23 participants. We observe that many participants lacked awareness of how VR collects data or the extent of passive tracking in VR. Based on these findings, we offer recommendations for VR permissions models and suggest directions for future work.

Fuzzing is a widely adopted technique for uncovering software vulnerabilities by generating random or mutated test inputs to trigger unexpected behavior. However, little is known about how developers actually use fuzzing tools in practice, the challenges they face, and where current tools fall short. This study investigates the human side of fuzzing via 18 semi-structured interviews with fuzzing users across diverse domains. These interviews explore participants' workflows, frustrations, and expectations around fuzzing, revealing critical usability gaps and design opportunities. Our results can inform the next generation of fuzzing tools to improve user experience, reduce manual effort, and enable more effective integration of fuzzing into real-world workflows.

Following the recent release of AI assistants, such as OpenAI's ChatGPT and GitHub Copilot, the software industry quickly utilized these tools for software development tasks, e.g., generating code or consulting AI for advice. While recent research has demonstrated that AI-generated code can contain security issues, how software professionals balance AI assistant usage and security remains unclear. This paper investigates how software professionals use AI assistants in secure software development, what security implications and considerations arise, and what impact they foresee on secure software development. We conducted 27 semi-structured interviews with software professionals, including software engineers, team leads, and security testers. We also reviewed 190 relevant Reddit posts and comments to gain insights into the current discourse surrounding AI assistants for software development. Our analysis of the interviews and Reddit posts finds that despite many security and quality concerns, participants widely use AI assistants for security-critical tasks, e.g., code generation, threat modeling, and vulnerability detection. Their overall mistrust leads to checking AI suggestions in similar ways to human code, although they expect improvements and, therefore, a heavier use for security tasks in the future. We conclude with recommendations for software professionals to critically check AI suggestions, AI creators to improve suggestion security and capabilities for ethical security tasks, and academic researchers to consider general-purpose AI in software development.

The Family Education Rights and Privacy Act (FERPA) is intended to protect student privacy, but has not adapted well to current technology. We consider a special class of student data: directory information. Unlike other FERPA-controlled data, directory information (e.g., student names, contact information, university affiliation) can be shared publicly online or by request without explicit permission.

Vulnerability discovery is an essential aspect of software security. Currently, the demand for security experts significantly exceeds the available vulnerability discovery workforce. Further, the existing vulnerability discovery workforce is highly homogeneous, dominated by white and Asian men. As such, one promising avenue for increasing the capacity of the vulnerability discovery community is through recruitment and retention from a broader population. Although significant prior research has explored the challenges of equity and inclusion in computing broadly, the competitive and frequently self-taught nature of vulnerability discovery work may create new variations on these challenges. This paper reports on a semi-structured interview study (N = 16) investigating how people from marginalized populations come to participate in vulnerability discovery, whether they feel welcomed by the vulnerability discovery community, and what challenges they face when joining the vulnerability discovery community. We find that members of marginalized populations face some unique challenges, while other challenges common in vulnerability discovery are exacerbated by marginalization.

Significant efforts have been investigated to develop machine learning (ML) based tools to support security operations. However, they still face key challenges in practice. A generally perceived weakness of machine learning is the lack of explanation, which motivates researchers to develop machine learning explanation techniques. However, it is not yet well understood how security practitioners perceive the benefits and pain points of machine learning and corresponding explanation methods in the context of security operations. To fill this gap and understand "what is needed", we conducted semi-structured interviews with 18 security practitioners with diverse roles, duties, and expertise. We find practitioners generally believe that ML tools should be used in conjunction with (instead of replacing) traditional rule-based methods. While ML’s output is perceived as difficult to reason, surprisingly, rule-based methods are not strictly easier to interpret. We also find that only few practitioners considered security (robustness to adversarial attacks) as a key factor for the choice of tools. Regarding ML explanations, while recognizing their values in model verification and understanding security events, practitioners also identify gaps between existing explanation methods and the needs of their downstream tasks. We collect and synthesize the suggestions from practitioners regarding explanation scheme designs, and discuss how future work can help to address these needs.

Software reverse engineering is a challenging and time consuming task. With the growing demand for reverse engineering in vulnerability discovery and malware analysis, manual reverse engineering cannot scale to meet the demand. There has been significant effort to develop automated tooling to support reverse engineers, but many reverse engineers report not using these tools. In this paper, we seek to understand whether this lack of use is an issue of usability. We performed a iterative open coding of 288 reverse engineering tools to identify common input and output methods, as well as whether the tools adhered to usability guidelines established in prior work. We found that most reverse engineering tools have limited interaction and usability support. However, usability issues vary between dynamic and static tools. Dynamic tools were less likely to provide easy-to-use interfaces, while static tools often did not allow reverse engineers to adjust the analysis. Based on our findings, we give recommendations for reverse engineering framework developers and suggest directions for future HCI research in reverse engineering.

This paper presents the results of in-depth study of 14 teams' development processes during a three-week undergraduate course organized around a secure coding competition. Contest participants were expected to first build code to a specification---emphasizing correctness, performance, and security---and then to find vulnerabilities in other teams' code while fixing discovered vulnerabilities in their own code. Our study aimed to understand why developers introduce different vulnerabilities, the ways they evaluate programs for vulnerabilities, and why different vulnerabilities are (not) found and (not) fixed. We used iterative open coding to systematically analyze contest data including code, commit messages, and team design documents. Our results point to the importance of existing best practices for secure development, the use of security tools, and development team organization.

Incident response playbooks provide step-by-step guidelines to help security operations personnel quickly respond to specific threat scenarios. Although playbooks are common in the security industry, they have not been empirically evaluated for effectiveness. This paper takes a first step toward measuring playbooks and the frameworks used to design them, using two studies conducted in an enterprise environment. In the first study, twelve security professionals created two playbooks each, using two standard playbook design frameworks; the resulting playbooks were evaluated by experts for accuracy. In the second, we observed five personnel using the created playbooks in no-notice threat exercises within a live security-operations center. We find that playbooks can help simplify and support incident response efforts. However, playbooks designed using the frameworks we examined often lack sufficient detail for real-world use, particularly for more junior technicians. We provide recommendations for improving playbooks, playbook frameworks, and organizational processes surrounding playbook use.

Reverse engineering is a complex task. As with many other expert tasks, reverse engineers rely on colleagues and the broader reverse engineering community to provide guidance and develop knowledge necessary to achieve their goals. For example, it is common for reverse engineers to reach out for help to understand and effectively use new tools. Thus far, however, there has been limited investigation of the way knowledge is developed in this community and new tools are adopted. This paper takes a first step toward understanding reverse engineering community dynamics around tool adoption, using the release of the National Security Agency's Ghidra reverse engineering framework as a point of focus. In this paper, we review discussions about Ghidra to identify what features reverse engineers are most interested in, how reverse engineers develop knowledge about Ghidra together online, and whether these dynamics differ between forums. In total, we analyze 1590 reverse engineering discussions between 688 reverse engineers over 3 forums (i.e., Twitter, Reddit, and StackExchange). Our results suggest reverse engineers are most interested in features that allow them to customize Ghidra. We also observe limited evidence of collective sensemaking on the forums, with few reverse engineers participating in multiple discussions threads and most acting as either knowledge producers or consumers. Finally, we found that the forums operated similarly, but Twitter was most often used to announce information (e.g., tutorial links, tool overviews, vulnerabilities in Ghidra) and reverse engineers used StackExchange mostly to get support for specific problems. Reddit acted as a middle option. Based on these results, we make recommendations to improve reverse engineering tool development, improve community participation during adoption, and suggest directions for future work.

Hacking exercises are a common tool for security education, but there is limited investigation of how they teach security concepts and whether they follow pedagogical best practices. This paper enumerates the pedagogical practices of 31 popular online hacking exercises. Specifically, we derive a set of pedagogical dimensions from the general learning sciences and educational literature, tailored to hacking exercises, and review whether and how each exercise implements each pedagogical dimension. In addition, we interview the organizers of 15 exercises to understand challenges and tradeoffs that may occur when choosing whether and how to implement each dimension.We found hacking exercises generally were tailored to students’ prior security experience and support learning by limiting extraneous load and establishing helpful online communities. Conversely, few exercises explicitly provide overarching conceptual structure or direct support for metacognition to help students transfer learned knowledge to new contexts. Immediate and tailored feedback and secure development practice were also uncommon. Additionally, we observed a tradeoff between providing realistic challenges and burdening students with extraneous cognitive load, with benefits and drawbacks at any point on this axis. Based on our results, we make suggestions for exercise improvement and future work to support organizers.

Security is an essential component of the software development lifecycle. Researchers and practitioners have developed educational interventions, guidelines, security analysis tools, and new APIs aimed at improving security. However, measuring any resulting improvement in secure development skill is challenging. As a proxy for skill, we propose to measure self-efficacy, which has been shown to correlate with skill in other contexts. Here, we present a validated scale measuring secure software-development self-efficacy (SSD-SES). We first reviewed popular secure-development frameworks and surveyed 22 secure-development experts to identify 58 unique tasks. Next, we asked 311 developers - over multiple rounds - to rate their skill at each task. We iteratively updated our questions to ensure they were easily understandable, showed adequate variance between participants, and demonstrated reliability. Our final 15-item scale contains two sub-scales measuring belief in ability to perform vulnerability identification and mitigation as well as security communications tasks.

Typical security contests focus on breaking or mitigating the impact of buggy systems. We present the Build-it, Break-it, Fix-it (BIBIFI) contest, which aims to assess the ability to securely build software, not just break it. In BIBIFI, teams build specified software with the goal of maximizing correctness, performance, and security. The latter is tested when teams attempt to break other teams’ submissions. Winners are chosen from among the best builders and the best breakers. BIBIFI was designed to be open-ended—teams can use any language, tool, process, and so on, that they like. As such, contest outcomes shed light on factors that correlate with successfully building secure software and breaking insecure software. We ran three contests involving a total of 156 teams and three different programming problems. Quantitative analysis from these contests found that the most efficient build-it submissions used C/C++, but submissions coded in a statically type safe language were 11× less likely to have a security flaw than C/C++ submissions. Break-it teams that were also successful build-it teams were significantly better at finding security bugs.

Many popular social networking and microblogging sites support verified accounts---user accounts that are deemed of public interest and whose owners have been authenticated by the site. Importantly, the content of messages contributed by verified account owners is not verified. Such messages may be factually correct, or not. This paper investigates whether users confuse authenticity with credibility by posing the question: Are users more likely to believe content from verified accounts than from non-verified accounts? We conduct two online studies, a year apart, with 748 and 2041 participants respectively, to assess how the presence or absence of verified account indicators influences users' perceptions of tweets. Surprisingly, across both studies, we find that---in the context of unfamiliar accounts---most users can effectively distinguish between authenticity and credibility. The presence or absence of an authenticity indicator has no significant effect on willingness to share a tweet or take action based on its contents.

Identifying security vulnerabilities in software is a critical task that requires significant human effort. Currently, vulnerability discovery is often the responsibility of software testers before release and white-hat hackers (often within bug bounty programs) afterward. This arrangement can be ad-hoc and far from ideal; for example, if testers could identify more vulnerabilities, software would be more secure at release time. Thus far, however, the processes used by each group - and how they compare to and interact with each other - have not been well studied. This paper takes a first step toward better understanding, and eventually improving, this ecosystem: we report on a semi-structured interview study (n=25) with both testers and hackers, focusing on how each group finds vulnerabilities, how they develop their skills, and the challenges they face. The results suggest that hackers and testers follow similar processes, but get different results due largely to differing experiences and therefore different underlying knowledge of security concepts. Based on these results, we provide recommendations to support improved security training for testers, better communication between hackers and developers, and smarter bug bounty policies to motivate hacker participation.

Android and other mobile operating systems ask users for authorization before allowing apps to access sensitive resources such as contacts and location. We hypothesize that such authorization systems could be improved by becoming more integrated with the app's user interface. In this paper, we conduct two studies to test our hypothesis. First, we use apptracer{}, a dynamic analysis tool we developed, to measure to what extent user interactions and sensitive resource use are related in existing apps. Second, we conduct an online survey to examine how different interactions with the UI affect users' expectations about whether an app accesses sensitive resources. Our results suggest that user interactions such as button clicks can be interpreted as authorization, reducing the need for separate requests; but that accesses not directly tied to user interactions should be separately authorized, possibly when apps are first launched.

Like computers, mobile devices are both used to commit and are the subject of crimes. Digital forensics collection and analysis techniques need to adapt to cope with the unique characteristics of mobile devices. Fortunately, the rapid adoption of such devices has also resulted in a relatively homogeneous set of software platforms. In this paper, we describe a general methodology for performing digital forensic data collection on Android-based devices. By re-purposing a special Android boot mode, comprehensive extraction of evidence, with minimal potential for data corruption or omission, is possible.