In this study, we take another look at 5 years of web censorship data gathered by the OpenNet Initiative in 77 countries using user-based testing with locally relevant content. Prior to our work, this data had been analyzed with little automation, focusing on what content had been blocked, rather than how blocking was carried out. In this study, we use more rigorous automation to obtain a longitudinal, global view of the technical means used for web censorship. We also identify blocking that had been missed in prior analyses. Our results point to considerable variability in the technologies used for web censorship, across countries, time, and types of content, and even across ISPs in the same country. In addition to characterizing web censorship in countries that, thus far, have eluded technical analysis, we also discuss the implications of our observations on the design of future network measurement platforms and circumvention technologies.

Products used for managing network traffic and restricting access to Web content represent a dual-use technology. While they were designed to improve performance and protect users from inappropriate content, these products are also used to censor the Web by authoritarian regimes around the globe. This dual use has not gone unnoticed, with Western governments placing restrictions on their export. Our contribution is to present methods for identifying installations of URL filtering products and confirming their use for censorship. We first present a methodology for identifying externally visible installations of URL filtering products in ISPs around the globe. Further, we leverage the fact that many of these products accept user-submitted sites for blocking to confirm that a specific URL filtering product is being used for censorship. Using this method, we are able to confirm the use of McAfee SmartFilter in Saudi Arabia and the United Arab Emirates (UAE) and Netsweeper in Qatar, the UAE, and Yemen. Our results show that these products are being used to block a range of content, including oppositional political speech, religious discussion and gay and lesbian material, speech generally protected by international human rights norms.