Many websites contain services from third parties. Misconfigurations of these services can lead to missing compliance with legal obligations and privacy risks for website users. Previous research indicates that one cause for such privacy issues is missing awareness. However, reasons for the missing awareness and other reasons for the prevalence of privacy issues are not widely researched; that includes website owners’ dealing with those issues. To shed light on the issue, we analyze 1043 responses from website owners to a notification about a privacy issue on their website using thematic analysis, following an exploratory and qualitative approach. Our analysis shows that, next to unawareness of the issue, incorrect technical implementation and ambiguous responsibilities are among the reasons for privacy issues. Also, website owners face different challenges, such as a lack of knowledge or slow organizational coordination and processes. In addition, our results show that the circumstances in which they operate their website influences how they act and what challenges they face. To illustrate these differences in website owners, we derive three personas from our thematic analysis: (1) the Ignorant Hobbyist, (2) the Busy Self-Employed, and (3) the Informed Multi-Stakeholder. These personas cover the majority of the aspects of the analyzed responses and represent the diversity of website owners and their backgrounds. Given the challenges and backgrounds of website owners, we discuss which prerequisites must be fulfilled to remediate privacy issues on websites. Finally, we present measures that support website owners in remediating privacy issues and show how to adapt these measures to the needs of different website owners. We hope that better support for website owners will also lead to better privacy for website visitors.