The mathematical study of voting, social choice theory , has traditionally only been applicable to choices among predetermined alternatives, but not to open-ended decisions such as collectively selecting a textual statement. We introduce generative social choice , a design methodology for open-ended democratic processes that combines the rigor of social choice theory with the capability of large language models to generate text and extrapolate preferences. Our framework divides the design of AI-augmented democratic processes into two components: first, proving that the process satisfies representation guarantees when given access to oracle queries; second, empirically validating that these queries can be approximately implemented using a large language model. We apply this framework to the problem of summarizing free-form opinions into a proportionally representative set of opinion statements; specifically, we develop a democratic process with representation guarantees and use this process to portray the opinions of participants in a survey about abortion policy. In a trial with 100 representative US residents, we find that 84 out of 100 participants feel “excellently” or “exceptionally” represented by the set of five statements we extracted.

The mathematical study of voting, social choice theory, has traditionally only been applicable to choices among a few predetermined alternatives, but not to open-ended decisions such as collectively selecting a textual statement. We introduce generative social choice, a design methodology for open-ended democratic processes that combines the rigor of social choice theory with the capability of large language models to generate text and extrapolate preferences. Our framework divides the design of AI-augmented democratic processes into two components: first, proving that the process satisfies representation guarantees when given access to oracle queries; second, empirically validating that these queries can be approximately implemented using a large language model. We apply this framework to the problem of summarizing free-form opinions into a proportionally representative slate of opinion statements; specifically, we develop a democratic process with representation guarantees and use this process to represent the opinions of participants in a survey about chatbot personalization. In a trial with 100 representative US residents, we find that 93 out of 100 participants feel "mostly" or "perfectly" represented by the slate of five statements we extracted. By providing rigorous guarantees through social choice, our work alleviates concerns about AI-driven democratic innovation and helps unlock its potential.

During the COVID-19 pandemic, many educators have had to rethink their methodology for summative assessment. Are timed and proctored exams appropriate---or even feasible---in this new open-internet, online learning environment? In this experience paper, we discuss our unique exams framework: our tool for generating exams that are uniquely identifiable but conceptually identical. In our university-level Probability for Computer Scientists Course, students completed unique exams generated from a common exam skeleton, with unique numeric variations per problem. With few deviations from the creation, administration, and grading processes of a traditional exam, our framework can provide a layer of security for both students and instructors about exam reliability for any classroom environment---in-person or online. In addition to sharing our experience designing unique exams, in this paper we also present a simple end-to-end tool and example question templates for different CS subjects that other instructors can adapt to their own courses.

Perceptual ad-blocking is a novel approach that detects online advertisements based on their visual content. Compared to traditional filter lists, the use of perceptual signals is believed to be less prone to an arms race with web publishers and ad networks. We demonstrate that this may not be the case. We describe attacks on multiple perceptual ad-blocking techniques, and unveil a new arms race that likely disfavors ad-blockers. Unexpectedly, perceptual ad-blocking can also introduce new vulnerabilities that let an attacker bypass web security boundaries and mount DDoS attacks. We first analyze the design space of perceptual ad-blockers and present a unified architecture that incorporates prior academic and commercial work. We then explore a variety of attacks on the ad-blocker's detection pipeline, that enable publishers or ad networks to evade or detect ad-blocking, and at times even abuse its high privilege level to bypass web security boundaries. On one hand, we show that perceptual ad-blocking must visually classify rendered web content to escape an arms race centered on obfuscation of page markup. On the other, we present a concrete set of attacks on visual ad-blockers by constructing adversarial examples in a real web page context. For seven ad-detectors, we create perturbed ads, ad-disclosure logos, and native web content that misleads perceptual ad-blocking with 100% success rates. In one of our attacks, we demonstrate how a malicious user can upload adversarial content, such as a perturbed image in a Facebook post, that fools the ad-blocker into removing another users' non-ad content. Moving beyond the Web and visual domain, we also build adversarial examples for AdblockRadio, an open source radio client that uses machine learning to detects ads in raw audio streams.

With the celebrated success of deep learning, some attempts to develop effective methods for detecting malicious PowerShell programs employ neural nets in a traditional natural language processing setup while others employ convolutional neural nets to detect obfuscated malicious commands at a character level. While these representations may express salient PowerShell properties, our hypothesis is that tools from static program analysis will be more effective. We propose a hybrid approach combining traditional program analysis (in the form of abstract syntax trees) and deep learning. This poster presents preliminary results of a fundamental step in our approach: learning embeddings for nodes of PowerShell ASTs. We classify malicious scripts by family type and explore embedded program vector representations.

Despite dramatic changes in technology over the past several years, educating young students in computer science at the elementary school level remains a challenge. Tackling this issue, we created 'Codester,' a novel Android application, as a tool to engage learners in the basics of computer science and to teach them computational thinking. Unlike other educational coding programs such as Scratch and Lego Mindstorms, our app is unique because it is mobile and allows users to learn a variety of concepts at a rapid rate. By running user studies for participants in grades 1-3 and 4-6, we quantified the effectiveness of Codester. Students were challenged and developed logic skills throughout the sessions. Using surveys, pretests, and posttests, we found an improvement in the five main computational thinking concepts that Codester teaches: sequencing, iteration, code reuse, decision-making, and logic. The participants especially showed an increased understanding of the code reuse concept: grade 1-3 students improved by 32% at the end of the program; grade 4-6 users improved by 43%. Additionally, the software and the tablet platform were attractive to the participants. Eighty percent of students stated that they enjoyed using the tablets to learn new material, and 72% said that they would like to continue learning with Codester. These figures show a promising increase of interest in coding and thus match the goal of our app: teach youngsters programming in a fun and challenging way. Please see www.codesterapp.com.