In this paper, we present and evaluate an automated pipeline for the large-scale analysis of corporate privacy policies. Organizations usually develop their privacy policies in isolation to best balance their business needs, user rights, as well as regulatory requirements. A wide-ranging and structured analysis of corporate privacy policies is essential to facilitate a deeper understanding of how organizations have balanced competing requirements. Our approach consists of a web crawler that can navigate to and scrape content from web pages that contain privacy policies, and a set of AI chatbot task prompts to process and extract structured/labeled annotations from the raw data. The analysis includes the types of collected user data, the purposes for which data is collected and processed, data retention and protection practices, and user rights and choices. Our validation shows that our annotations are highly accurate and consistent. We use this architecture to gather data on the privacy policies of companies in the Russell 3000 index, resulting in hundreds of thousands of annotations across all categories. Analysis of the resulting data allows us to obtain unique insights into the state of the privacy policy ecosystem as a whole.

Distributed Denial of Service (DDoS) attacks based on Network Time Protocol (NTP) amplification, which became prominent in December 2013, have received significant global attention. We chronicle how this attack rapidly rose from obscurity to become the dominant large DDoS vector. Via the lens of five distinct datasets, we characterize the advent and evolution of these attacks. Through a dataset that measures a large fraction of global Internet traffic, we show a three order of magnitude rise in NTP. Using a large darknet, we observe a similar rise in global scanning activity, both malicious and research. We then dissect an active probing dataset, which reveals that the pool of amplifiers totaled 2.2M unique IPs and includes a small number of "mega amplifiers," servers that replied to a single tiny probe packet with gigabytes of data. This dataset also allows us, for the first time, to analyze global DDoS attack victims (including ports attacked) and incidents, where we show 437K unique IPs targeted with at least 3 trillion packets, totaling more than a petabyte. Finally, ISP datasets shed light on the local impact of these attacks. In aggregate, we show the magnitude of this major Internet threat, the community's response, and the effect of that response.

We report the results of a study to collect and analyze IPv6 Internet background radiation. This study, the largest of its kind, collects unclaimed traffic on the IPv6 Internet by announcing five large covering prefixes; these cover the majority of allocated IPv6 space on today's Internet. Our analysis characterizes the nature of this traffic across regions, over time, and by the allocation and routing status of the intended destinations, which we show help to identify the causes of this traffic. We compare results to unclaimed traffic in IPv4, and highlight case studies that explain a large fraction of the data or highlight notable properties. We describe how announced covering prefixes differ from traditional network telescopes, and show how this technique can help both network operators and the research community identify additional potential issues and misconfigurations in this critical Internet transition period.

The monitoring of packets destined for routeable, yet unused, Internet addresses has proved to be a useful technique for measuring a variety of specific Internet phenomenon (e.g., worms, DDoS). In 2004, Pang et al. stepped beyond these targeted uses and provided one of the first generic characterizations of this non-productive traffic, demonstrating both its significant size and diversity. However, the six years that followed this study have seen tremendous changes in both the types of malicious activity on the Internet and the quantity and quality of unused address space. In this paper, we revisit the state of Internet ”background radiation ” through the lens of two unique data-sets: a five-year collection from a single unused /8 network block, and week-long collections from three recently allocated /8 network blocks. Through

While BGP routing datasets, consisting of raw routing data, are freely available and easy to obtain, extracting any useful information is tedious. Currently, researcher and network operators implement their own custom data processing tools and scripts. A single tool that provides easy access to the information within large raw BGP datasets could be used by both communities to avoid re-writing these tools each time. Moreover, providing not just raw BGP messages, but some commonly used summary statistics as well can help guide deeper custom analyses. Based on these observations this paper describes the first steps towards building a scalable tool. We describe the various techniques and algorithms we have used to build an efficient generic tool called BGP-Inspect. When dealing with large datasets, dataset size, lookup speed, and data processing time are the most challenging issues. We describe our implementations of chunked compressed files and B+ tree indices that attempt to address these issues. We then provide an evaluation of our implementations. Finally, we provide some example scenarios and case studies where BGP-Inspect can provide useful insight into the management and operation of complex BGP based networks. An efficient and flexible back-end custom BGP message database, coupled with an intuitive and easy to use Web-based query front-end makes BGP-Inspect a unique and powerful tool

In this paper we describe a set of visualization techniques that can help the task of operating and managing a network by representing network traffic information in a concise and intuitive manner. We have implemented these visualization techniques in Flamingo, a software tool that can be used to explore Internet traffic flow data. Flamingo is able to process live Netflow data in real-time and present a set of interactive visualizations and associated manipulation tools that can help users in network data analysis. Flamingo is comprised of a server and a client component. The Flamingo server is responsible for receiving raw Netflow feeds from devices in the network that can sample traffic, and then sending processed information to the client for display. The Flamingo client receives data from the server and provides concise intuitive data visualizations, 3D space navigation, as well as filtering capabilities that can help the operator to extract or monitor specific information of interest. Flamingo also supports a playback mode which allows users to select specific historical Netflow data for visualization. We illustrate with the help of simple examples, based on traffic data from a busy Internet backbone router, how Flamingo can be used to perform network monitoring tasks as well as network security related data forensics.

Extracting specific and relevant information regarding the Internet's BGP routing topology is a challenging task. In this paper we present a set of techniques that can be used to visualize various aspects of the Internet topology. We have implemented our visualization techniques in a unique tool called VAST (Visualizing Autonomous System Topology). With the help of simple illustrative examples we describe how our visualizations allow security researchers to extract relevant information quickly from raw BGP routing datasets. VAST provides visualizations that represent information about both the overall topological properties of the Internet as well as individual Autonomous System (AS) behavior.