We analyze the system Amazon deploys on the US “amazon.com” storefront to restrict shipments of certain products to other regions in the world. We found 17,842 products that Amazon restricted from being shipped to at least one world region. While many of the shipping restrictions are related to regulations involving WiFi, car seats, and other heavily regulated product categories, the most common product category restricted by Amazon in our study was books. Banned books were largely related to LGBTIQ, the occult, erotica, Christianity, and health and wellness. The regions affected by this censorship were the UAE, Saudi Arabia, and many other Middle Eastern countries as well as Brunei Darussalam, Papua New Guinea, Seychelles, and Zambia. In our Common Crawl test sample, Amazon restricted shipment of over 1.1% of the books sold on amazon.com to at least one of these regions. We identified three major blocklists which Amazon assigns to different regions. In numerous cases, Amazon’s restrictions were either overly broad or miscategorized. Examples include the restriction of books relating to breast cancer, recipe books invoking “food porn” euphemisms, and Nietzsche’s Gay Science. To justify why restricted products cannot be shipped, Amazon used varying and often misleading error messages such as by conveying that an item is temporarily out of stock. We reported our findings to Amazon. A year later, we found that Amazon reduced its use of misleading error messages and had eliminated many of its false positive restrictions. Although we cannot conclusively link Amazon’s improvements to our reporting, we believe that our findings serve as evidence that measurement studies can effect change in problematic company behavior.

Advances in networking and firewall technology have led to the emergence of network censorship devices that can perform large-scale, highly-performant content blocking. While such devices have proliferated, techniques to locate, identify, and understand them are still limited, require cumbersome manual effort, and are developed on a case-by-case basis.

In this study, we take another look at 5 years of web censorship data gathered by the OpenNet Initiative in 77 countries using user-based testing with locally relevant content. Prior to our work, this data had been analyzed with little automation, focusing on what content had been blocked, rather than how blocking was carried out. In this study, we use more rigorous automation to obtain a longitudinal, global view of the technical means used for web censorship. We also identify blocking that had been missed in prior analyses. Our results point to considerable variability in the technologies used for web censorship, across countries, time, and types of content, and even across ISPs in the same country. In addition to characterizing web censorship in countries that, thus far, have eluded technical analysis, we also discuss the implications of our observations on the design of future network measurement platforms and circumvention technologies.

Products used for managing network traffic and restricting access to Web content represent a dual-use technology. While they were designed to improve performance and protect users from inappropriate content, these products are also used to censor the Web by authoritarian regimes around the globe. This dual use has not gone unnoticed, with Western governments placing restrictions on their export. Our contribution is to present methods for identifying installations of URL filtering products and confirming their use for censorship. We first present a methodology for identifying externally visible installations of URL filtering products in ISPs around the globe. Further, we leverage the fact that many of these products accept user-submitted sites for blocking to confirm that a specific URL filtering product is being used for censorship. Using this method, we are able to confirm the use of McAfee SmartFilter in Saudi Arabia and the United Arab Emirates (UAE) and Netsweeper in Qatar, the UAE, and Yemen. Our results show that these products are being used to block a range of content, including oppositional political speech, religious discussion and gay and lesbian material, speech generally protected by international human rights norms.