Free and open source social platform software has dramatically lowered the barrier to entry for anyone to set up and administer their own social network. This new population of social network administrators thus assume data management responsibilities for sociotechnical systems. Administrators have the power to customize this software, including data collection and data retention, potentially leading to radically different privacy policies. To better understand the characteristics — e.g., the variability, prohibitions, and permissions — of privacy policies on these new social networking platforms, we have conducted a case study of Mastodon. We performed a text analysis of 351 privacy policies and a survey of 104 Mastodon administrators. While most administrators used the default policy that ships with the Mastodon software, we observed that approximately ten percent of our sample tailored their privacy policies to their instances and that some administrators conflated codes of conduct with privacy policies. Our findings suggest the existing market-based individualistic frameworks for thinking about privacy policies do not adequately address this emerging community.

Online experiments have become a ubiquitous aspect of design and engineering processes within Internet firms. As the scale of experiments has grown, so has the complexity of their design and implementation. In response, firms have developed software frameworks for designing and deploying online experiments. Ensuring that experiments in these frameworks are correctly designed and that their results are trustworthy---referred to as internal validity---can be difficult. Currently, verifying internal validity requires manual inspection by someone with substantial expertise in experimental design. We present the first approach for statically checking the internal validity of online experiments. Our checks are based on well-known problems that arise in experimental design and causal inference. Our analyses target PlanOut, a widely deployed, open-source experimentation framework that uses a domain-specific language to specify and run complex experiments. We have built a tool called PlanAlyzer that checks PlanOut programs for a variety of threats to internal validity, including failures of randomization, treatment assignment, and causal sufficiency. PlanAlyzer uses its analyses to automatically generate contrasts, a key type of information required to perform valid statistical analyses over the results of these experiments. We demonstrate PlanAlyzer's utility on a corpus of PlanOut scripts deployed in production at Facebook, and we evaluate its ability to identify threats to validity on a mutated subset of this corpus. PlanAlyzer has both precision and recall of 92% on the mutated corpus, and 82% of the contrasts it generates match hand-specified data.

Surveys can be viewed as programs, complete with logic, control flow, and bugs. Word choice or the order in which questions are asked can unintentionally bias responses. Vague, confusing, or intrusive questions can cause respondents to abandon a survey. Surveys can also have runtime errors: inattentive respondents can taint results. This effect is especially problematic when deploying surveys in uncontrolled settings, such as on the web or via crowdsourcing platforms. Because the results of surveys drive business decisions and inform scientific conclusions, it is crucial to make sure they are correct.