Apple introduced privacy labels in Dec. 2020 as a way for developers to report the privacy behaviors of their apps. While Apple does not validate labels, they also require developers to provide a privacy policy, which offers an important comparison point. In this paper, we fine-tuned BERT-based language models to extract privacy policy features for 474,669 apps on the iOS App Store, comparing the output to the privacy labels. We identify discrepancies between the policies and the labels, particularly as they relate to data collected linked to users. We find that 228K apps' privacy policies may indicate data collection linked to users than what is reported in the privacy labels. More alarming, a large number (97%) of the apps with a Data Not Collected privacy label have a privacy policy indicating otherwise. We provide insights into potential sources for discrepancies, including the use of templates and confusion around Apple's definitions and requirements. These results suggest that significant work is still needed to help developers more accurately label their apps. Our system can be incorporated as a first-order check to inform developers when privacy labels are possibly misapplied.

Data dashboards are designed to help users manage data collected about them. However, prior work showed that exposure to some dashboards, notably Google’s My Activity dashboard, results in significant decreases in perceived concern and increases in perceived benefit from data collection, contrary to expectations. We theorize that this result is due to the fact that data dashboards currently do not sufficiently “connect the dots” of the data food chain, that is, by connecting data collection with the use of that data. To evaluate this, we designed a study where participants assigned advertising interest labels to their own real activities, effectively acting as a behavioral advertising engine to “connect the dots.” When comparing pre- and post-labeling task responses, we find no significant difference in concern with Google’s data collection practices, which indicates that participants’ priors are maintained after more exposure to the data food chain (differing from prior work), suggesting that data dashboards that offer deeper perspectives of how data collection is used have potential. However, these gains are offset when participants are exposed to their true interest labels inferred by Google. Concern for data collection dropped significantly as participants viewed Google’s labeling as generic compared to their own more specific labeling. This presents a possible new paradox that must be overcome when designing data dashboards, the generic paradox, which occurs when users misalign individual, generic inferences from collected data as benign compared to the totality and specificity of many generic inferences made about them.

This work reports on a large scale, longitudinal analysis of the adoption dynamics of privacy labels in the iOS App Store, measuring this first-of-its kind ecosystem as it reaches maturity over two and a half years after launching in December 2020. The motivation is to shed light on the factors affecting the shifts in privacy labels and provide insights into how and when an app's label changes. By collecting nearly weekly snapshots of over 1.6 million apps for over a year, we analyze the dynamics of privacy label adoption and the accuracy of reported labels. Our analysis of 74.5% of apps having labels after two years provides important context into this mature ecosystem where labels are becoming the standard. However, we find compelling evidence that labels may not fully capture behavior, as 28.9% of apps indicate no data collection and distributions differ between voluntary versus mandatory adoptions. Once set, labels rarely change but additions reflect more data collection. In addition to our measurement, we also plan to release a new (and growing) data set that can be used by future researchers.