Answering end user security questions is challenging. While large language models (LLMs) like GPT, Llama, and Gemini are far from error-free, they have shown promise in answering a variety of questions outside of security. We qualitatively evaluated responses from three popular LLMs to 900 systematically collected security questions in the first such evaluation in the area of end user security. While LLMs demonstrate broad generalist “knowledge” of end user security information, there are patterns of errors and limitations across LLMs-including stale, inaccurate, and incomplete answers, as well as indirect or unresponsive communication styles-which negatively impact the user experience. Based on these patterns, we suggest directions for improved model development and recommend user strategies for interacting with LLMs when seeking assistance with security.

Digital financial services can introduce new digital-safety risks for users, particularly survivors of intimate partner financial abuse (IPFA). To offer improved support for such users, a comprehensive understanding of their support needs and the barriers they face to redress by financial institutions is essential. Drawing from a dataset of 2.7 million customer complaints, we implement a bespoke workflow that utilizes language-modeling techniques and expert human review to identify complaints describing IPFA. Our mixed-method analysis provides insight into the most common digital financial products involved in these attacks, and the barriers consumers report encountering when doing so. Our contributions are twofold; we offer the first human-labeled dataset for this overlooked harm and provide practical implications for technical practice, research, and design for better supporting and protecting survivors of IPFA.