Online targeted ads are those shown only to certain users based on interests, demographics, or behaviors. Because targeted ads raise many privacy concerns, many platforms provide ad transparency systems (ATSs) to inform users about this practice. To better understand what current ATSs are communicating to users—and how—we first taxonomized the design and content of 22 of the most popular English-language websites' ATSs as presented to users in the United States. We found substantial differences across ATSs in both the prevalence of transparency-enhancing features (e.g., whether they show users what has been inferred about them) and the presentation of information (e.g., the terminology used, where settings are located). Across all platforms, however, we observed consistent ambiguity about what data is used to target ads and the actual impact of altering settings. To gauge how these different design choices impact users, we conducted an online user study in which 198 participants used their own account to explore the ATS of one of eight representative platforms. We found that many of the questions participants hoped the ATS would answer remained unanswered after exploring the ATS. More broadly, participants found current ATSs simultaneously complex and lacking key details. We pinpoint ATS design decisions that best support users.

Current algorithmic fairness tools focus on auditing completed models, neglecting the potential downstream impacts of iterative decisions about cleaning data and training machine learning models. In response, we developed Retrograde, a JupyterLab environment extension for Python that generates real-time, contextual notifications for data scientists about decisions they are making regarding protected classes, proxy variables, missing data, and demographic differences in model performance. Our novel framework uses automated code analysis to trace data provenance in JupyterLab, enabling these notifications. In a between-subjects online experiment, 51 data scientists constructed loan-decision models with Retrograde providing notifications continuously throughout the process, only at the end, or never. Retrograde’s notifications successfully nudged participants to account for missing data, avoid using protected classes as predictors, minimize demographic differences in model performance, and exhibit healthy skepticism about their models.

The ubiquity of self-tracking devices and smartphone apps has empowered people to collect data about themselves and try to self-improve. However, people with little to no personal analytics experience may not be able to analyze data or run experiments on their own (self-experiments). To lower the barrier to intervention-based self-experimentation, we developed an app called Self-E, which guides users through the experiment. We conducted a 2-week diary study with 16 participants from the local population and a second study with a more advanced group of users to investigate how they perceive and carry out self-experiments with the help of Self-E, and what challenges they face. We find that users are influenced by their preconceived notions of how healthy a given behavior is, making it difficult to follow Self-E’s directions and trusting its results. We present suggestions to overcome this challenge, such as by incorporating empathy and scaffolding in the system.

Research to Practice Full Paper: The reliance on undergraduate students to take on key aspects of college level courses, such as grading, holding office hours, and facilitating small group work, has been growing with the increased enrollment in many introductory STEM courses. There is an urgent need to understand how these undergraduate teaching assistants (UTAs) approach and engage their roles in the wake of the increasing dependency on these students to facilitate learning within courses. Over the past few years at Brown University, and many others, UTAs have been hired at a growing rate and have been expected to take on new responsibilities. However, institutional policies and guidelines have not changed to reflect these new conditions, raising the question as to what are points of tension in the UTA experience and what are the causes of such tensions. This paper aims to address this concern by identifying and analyzing these tensions that arise as UTAs work to be successful educators.To further understand the tensions and pressures experienced by these student workers, we interviewed UTAs in engineering, physics, computer science, and one UTA in engineering and computer science (n=5). The departments vary according to the size and scope of their respective UTA programs. We use the framework of Activity Systems Analysis (ASA) in an effort to identify areas of tension and improvement in current teaching and support practices. The ASA framework allows researchers to narrow their focus on human activity and experiences, in this case those of the UTA, while not losing sight of the cultural and historical contexts in which this activity and these experiences take place. The framework also allows for the identification of systemic tensions that tie human activities and environments to undesired outcomes at an individual or systemic level. We identify tensions from the UTA experiences, provide recommendations for ways to better support and develop STEM UTAs, and identify future areas of research.