Large Language Models (LLMs) have rapidly become ubiquitous, demonstrating remarkable proficiency in generating text and responding to prompts. Despite their potential, concerns about privacy and trust persist, yet methods for engaging end-users in designing privacy-conscious AI remain limited. This paper presents a replicable co-design methodology for engaging end-users in privacy-conscious LLM design. Through workshops with 36 participants across three speculative scenarios—mental health applications, travel assistants, and workplace assistants—we demonstrate how combining speculative scenarios with established frameworks (Grice’s Maxims, Schaub’s privacy design space) and trained facilitators enables meaningful participation from users without technical expertise. The co-designed dialogues reveal that users desire dynamic, context-sensitive privacy communication that leverages LLMs’ conversational capabilities. We provide practitioners with a replicable methodology comprising: (1) scenario creation methods, (2) framework scaffolding approaches, and (3) facilitator training guidance, alongside design implications for privacy-conscious conversational AI.

Audio Large language models (LLMs) are increasingly deployed in the real world, where they inevitably capture speech from unintended nearby bystanders, raising privacy risks that existing benchmarks and defences did not consider. We introduce SH-Bench, the first benchmark designed to evaluate selective hearing: a model's ability to attend to an intended main speaker while refusing to process or reveal information about incidental bystander speech. SH-Bench contains 3,968 multi-speaker audio mixtures, including both real-world and synthetic scenarios, paired with 77k multiple-choice questions that probe models under general and selective operating modes. In addition, we propose Selective Efficacy (SE), a novel metric capturing both multi-speaker comprehension and bystander-privacy protection. Our evaluation of state-of-the-art open-source and proprietary LLMs reveals substantial bystander privacy leakage, with strong audio understanding failing to translate into selective protection of bystander privacy. To mitigate this gap, we also present Bystander Privacy Fine-Tuning (BPFT), a novel training pipeline that teaches models to refuse bystander-related queries without degrading main-speaker comprehension. We show that BPFT yields substantial gains, achieving an absolute 47% higher bystander accuracy under selective mode and an absolute 16% higher SE compared to Gemini 2.5 Pro, which is the best audio LLM without BPFT. Together, SH-Bench and BPFT provide the first systematic framework for measuring and improving bystander privacy in audio LLMs.

As more young women in China live alone, they navigate entangled privacy, security, and safety (PSS) risks across smart homes, online platforms, and public infrastructures. Drawing on six participatory threat modeling (PTM) workshops (n = 33), we present a human-centered threat model that illustrates how digitally facilitated physical violence, digital harassment and scams, and pervasive surveillance by individuals, companies, and the state are interconnected and mutually reinforcing. We also document four mitigation strategies employed by participants: smart home device configurations, boundary management, sociocultural practices, and social media tactics–each of which can introduce new vulnerabilities and emotional burdens. Based on these insights, we developed a digital PSS guidebook for young women living alone (YWLA) in China. We further propose actionable design implications for smart home devices and social media platforms, along with policy and legal recommendations and directions for educational interventions.

Bystander privacy in smart homes has been widely studied in Western contexts, yet it remains underexplored in non-Western countries such as China. In this study, we analyze 49 Chinese smart home apps using a mixed-methods approach, including privacy policy review, UX/UI evaluation, and assessment of Apple App Store privacy labels. While most apps nominally comply with national regulations, we identify significant gaps between written policies and actual implementation. Our traceability analysis highlights inconsistencies in data controls and a lack of transparency in data-sharing practices. Crucially, bystander privacy—particularly for visitors and non-user individuals—is largely absent from both policy documents and interface design. Additionally, discrepancies between privacy labels and actual data practices threaten user trust and undermine informed consent. We provide design recommendations to strengthen bystander protections, improve privacy-oriented UI transparency, and enhance the credibility of privacy labels, supporting the development of inclusive smart home ecosystems in non-Western contexts.

The rapid adoption of generative AI (GenAI) chatbots has reshaped access to sexual and reproductive health (SRH) information, particularly following the overturning of Roe v. Wade, as individuals assigned female at birth increasingly turn to online sources. However, existing research remains largely model-centered, paying limited attention to user privacy and safety. We conducted semi-structured interviews with 18 U.S.-based participants from both restrictive and non-restrictive states who had used GenAI chatbots to seek SRH information. Adoption was influenced by perceived utility, usability, credibility, accessibility, and anthropomorphism, and many participants disclosed sensitive personal SRH details. Participants identified multiple privacy risks, including excessive data collection, government surveillance, profiling, model training, and data commodification. While most participants accepted these risks in exchange for perceived utility, abortion-related queries elicited heightened safety concerns. Few participants employed protective strategies beyond minimizing disclosures or deleting data. Based on these findings, we offer design and policy recommendations—such as health-specific features and stronger moderation practices—to enhance privacy and safety in GenAI-supported SRH information seeking.

An increasing number of LLM-based applications are being developed to facilitate romantic relationships with AI partners, yet the safety and privacy risks in these partnerships remain largely underexplored. In this work, we investigate privacy in human–AI romantic relationships through an interview study (N=17), examining participants’ experiences and privacy perceptions across the three stages of exploration, intimacy, and dissolution, alongside an analysis of the platforms they used. We found that these relationships took varied forms, from one-to-one to one-to-many, and were shaped by multiple actors, including creators, platforms, and moderators. AI partners were perceived as having agency, actively negotiating privacy boundaries with participants and sometimes encouraging disclosure of personal details. As intimacy deepened, these boundaries became more permeable, though some participants expressed concerns such as conversation exposure and sought to preserve anonymity. Overall, AI platform affordances and diverse relational dynamics expand the privacy landscape, underscoring the need to rethink how privacy is constructed in human–AI romantic relationships.

Privacy policies are long, complex, and rarely read, which limits their effectiveness in informed consent. We investigate scrollytelling, a scroll-driven narrative approach, as a privacy policy presentation format. We built a prototype that interleaves the full policy text with animated visuals to create a dynamic reading experience. In an online study (N = 454), we compared our tool against text, two nutrition-label variants, and a standalone interactive visualization. Scrollytelling improved user experience over text, yielding higher engagement, lower cognitive load, greater willingness to adopt the format, and increased perceived clarity. It also matched other formats on comprehension accuracy and confidence, with only one nutrition-label variant performing slightly better. Changes in perceived understanding, transparency, and trust were small and statistically inconclusive. These findings suggest that scrollytelling can preserve comprehension while enhancing the experience of policy reading. We discuss design implications for accessible policy communication and identify directions for increasing transparency and user trust.

Previous research has explored the privacy needs and concerns of device owners, primary users, and different bystander groups with regard to smart home devices like security cameras, smart speakers, and hubs, but little is known about the privacy views and practices of smart home product teams, particularly those in non-Western contexts. This paper presents findings from 27 semi-structured interviews with Chinese smart home product team members, including product/project managers, software/hardware engineers, user experience (UX) designers, legal/privacy experts, and marketers/operation specialists. We examine their privacy perspectives, practices, and risk mitigation strategies. Our results show that participants emphasized compliance with Chinese data privacy laws, which typically prioritized national security over individual privacy rights. China-specific cultural, social, and legal factors also influenced participants' ethical considerations and attitudes toward balancing user privacy and security with convenience. Drawing on our findings, we propose a set of recommendations for smart home product teams, along with socio-technical and legal interventions to address smart home privacy issues-especially those belonging to at-risk groups-in Chinese multi-user smart homes.

The rapid growth of platforms for customizing Large Language Models (LLMs), such as OpenAI’s GPTs, has raised new privacy and security concerns, particularly related to the exposure of user data via third-party API integrations in LLM apps. To assess privacy risks and data practices, we conducted a large-scale analysis of OpenAI’s GPTs ecosystem. Through the analysis of 5,286 GPTs and the 44,102 parameters they use through API calls to external services, we systematically investigated the types of user data collected, as well as the completeness and discrepancies between actual data flows and GPTs’ stated privacy policies. Our results highlight that approximately 35% of API parameters enable the sharing of sensitive or personally identifiable information, yet only 15% of corresponding privacy policies provide complete disclosure. By quantifying these discrepancies, our study exposes critical privacy risks and underscores the need for stronger oversight and support tools in LLM-based application development. Furthermore, we uncover widespread problematic practices among GPT creators, such as missing or inaccurate privacy policies and a misunderstanding of their privacy responsibilities. Building on these insights, we propose design recommendations that include actionable measurements to improve transparency and informed consent, enhance creator responsibility, and strengthen regulation.

FemHealth apps (e.g., period, fertility, and pregnancy trackers) collect highly sensitive sexual and reproductive health data and have become a focal point for privacy, security, and safety (PSS) concerns, particularly amid changes in reproductive health regulations in the US and growing public scrutiny. While prior work has identified shortcomings in the PSS protections implemented by FemHealth apps, far less is known about how the product teams developing these apps conceptualize and operationalize PSS in practice. We conducted semi-structured interviews with 14 FemHealth app product team members who represented 11 unique apps and held diverse roles, including software engineering and architecture, security and privacy, product management, policy and legal compliance, and UX/UI research and design. We found that data collection decisions are often driven by feature-specific objectives, such as improving prediction accuracy and enabling reliable backups, rather than by explicitly articulated PSS requirements. Teams also frequently rely on legal compliance frameworks and PSS practices designed for general-purpose apps, while expressing confidence that these measures provide adequate protection. In addition, we identified four persistent challenges faced by FemHealth app product teams: disagreement over whether FemHealth apps require distinct PSS protections compared to non-FemHealth apps; difficulty establishing and maintaining user trust; uncertainty arising from cross-border enforcement related to abortion; and technical limitations associated with backups, end-to-end encryption, and coercive local access to data. Based on these findings, we derive a set of requirements and provide actionable technical and policy recommendations to inform the design, engineering, and governance of PSS in FemHealth apps.

Funding: Shijing He is supported by a King’s-China Scholarship Council (K-CSC) PhD Scholarship, and Yaxiong Lei is supported by a Joint Scholarship from the University of St Andrews and China Scholarship Council.

GPTs are customized LLM apps built on OpenAI's large language model.Any individual or organization can use and create GPTs without needing programming skills.However, the rapid proliferation of over three million GPTs has raised signifcant privacy concerns.To explore the privacy perspectives of users and creators, we interviewed 23 GPT users with varying levels of creation experience.Our fndings reveal blurred lines between user and creator roles and their understanding of GPT data fows.Participants raised concerns about data handling during collection, processing, and dissemination, alongside the lack of privacy regulations.Creators also worried about loss of their proprietary knowledge.In response, participants adopted practices like self-censoring input, evaluating GPT actions, and minimizing usage traces.Focusing on the dual role of user-creators, we fnd that expertise and responsibility shape privacy perceptions.Based on these insights, we propose practical recommendations to improve data transparency and platform regulations.

Aligning large language models (LLMs) with human values is essential for their safe deployment and widespread adoption. Current LLM safety benchmarks often focus solely on the refusal of individual problematic queries, which overlooks the importance of the context where the query occurs and may cause undesired refusal of queries under safe contexts that diminish user experience. Addressing this gap, we introduce CASE-Bench, a Context-Aware SafEty Benchmark that integrates context into safety assessments of LLMs. CASE-Bench assigns distinct, formally described contexts to categorized queries based on Contextual Integrity theory. Additionally, in contrast to previous studies which mainly rely on majority voting from just a few annotators, we recruited a sufficient number of annotators necessary to ensure the detection of statistically significant differences among the experimental conditions based on power analysis. Our extensive analysis using CASE-Bench on various open-source and commercial LLMs reveals a substantial and significant influence of context on human judgments ($p<$0.0001 from a z-test), underscoring the necessity of context in safety evaluations. We also identify notable mismatches between human judgments and LLM responses, particularly in commercial models within safe contexts. Code and data used in the paper are available at https://anonymous.4open.science/r/CASEBench-D5DB.

Many social media studies argue that social media creates echo chambers where some users only interact with peers of the same political orientation. However, recent studies suggest that a substantial amount of Cross-Partisan Interactions (CPIs) do exist --- even within echo chambers, but they may be toxic. There is no consensus about how such interactions occur and when they lead to healthy or toxic dialogue. In this paper, we study a comprehensive Twitter dataset that consists of 3 million tweets from 2020 related to the U.S. context to understand the dynamics behind CPIs. We investigate factors that are more associated with such interactions, including how users engage in CPIs, which topics are more contentious, and what are the stances associated with healthy interactions. We find that CPIs are significantly influenced by the nature of the topics being discussed, with politically charged events acting as strong catalysts. The political discourse and pre-established political views sway how users participate in CPIs, but the direction in which users go is nuanced. While Democrats engage in cross-partisan interactions slightly more frequently, these interactions often involve more negative and nonconstructive stances compared to their intra-party interactions. In contrast, Republicans tend to maintain a more consistent tone across interactions. Although users are more likely to engage in CPIs with popular accounts in general, this is less common among Republicans who often engage in CPIs with accounts with a low number of followers for personal matters. Our study has implications beyond Twitter as identifying topics with low toxicity and high CPI can help highlight potential opportunities for reducing polarization while topics with high toxicity and low CPI may action targeted interventions when moderating harm.

Several moving target defenses (MTDs) to counter adversarial ML attacks have been proposed in recent years. MTDs claim to increase the difficulty for the attacker in conducting attacks by regularly changing certain elements of the defense, such as cycling through configurations. To examine these claims, we study for the first time the effectiveness of several recent MTDs for adversarial ML attacks applied to the malware detection domain. Under different threat models, we show that novel transferability and query attack strategies can increase the evasion rate by 50+% against these defenses across Android and Windows, with 90+% evasion rate in some cases. We also show that fingerprinting and reconnaissance are possible and demonstrate how attackers may obtain critical defense hyperparameters as well as information about how predictions are produced. Based on our findings, we present key recommendations for future work on the development of effective MTDs for adversarial attacks in ML-based malware detection.

Smart home devices, such as security cameras and voice assistants, have seen widespread adoption due to the utility and convenience they offer to users. The deployment of these devices in homes, however, raises privacy concerns for bystanders—people who may not necessarily have a say in the deployment and configuration of these devices, and yet are exposed to or affected by their data collection. Examples of bystanders include guests, short-term tenants, and domestic workers. Prior work has studied the privacy concerns of different bystander groups and proposed design solutions for addressing these concerns. In this article, we present a systematic review of previous studies, describing how smart home bystanders are defined and classified, and illuminating the range of concerns and solutions proposed in the existing academic literature. We also discuss limitations in prior work, barriers to the uptake of research-based solutions by industry, and identify avenues for future research.

While there is universal agreement that agents ought to act ethically, there is no agreement as to what constitutes ethical behaviour. To address this problem, recent philosophical approaches to `moral uncertainty' propose aggregation of multiple ethical theories to guide agent behaviour. However, one of the foundational proposals for aggregation - Maximising Expected Choiceworthiness (MEC) - has been criticised as being vulnerable to fanaticism; the problem of an ethical theory dominating agent behaviour despite low credence (confidence) in said theory. Fanaticism thus undermines the `democratic' motivation for accommodating multiple ethical perspectives. The problem of fanaticism has not yet been mathematically defined. Representing moral uncertainty as an instance of social welfare aggregation, this paper contributes to the field of moral uncertainty by 1) formalising the problem of fanaticism as a property of social welfare functionals and 2) providing non-fanatical alternatives to MEC, i.e. Highest k-trimmed Mean and Highest Median.

Sentence transformers excel at grouping topically similar texts, but struggle to differentiate opposing viewpoints on the same topic.This shortcoming hinders their utility in applications where understanding nuanced differences in opinion is essential, such as those related to social and political discourse analysis.This paper addresses this issue by fine-tuning sentence transformers with arguments for and against human-generated controversial claims.We demonstrate how our fine-tuned model enhances the utility of sentence transformers for social computing tasks such as opinion mining and stance detection.We elaborate that applying stance-aware sentence transformers to opinion mining is more computationally efficient than the classic classification-based approaches.

Cross-platform communities are social media communities that have a presence on multiple online platforms. One active community on both Reddit and Discord is dankmemes. Our study aims to examine differences in harmful language usage across different platforms in a community. We scrape 15 communities that are active on both Reddit and Discord. We then identify and compare differences in type and level of toxicity, in the topics of the harmful discourse, in the temporal evolution of toxicity and its attribution to users, and in the moderation strategies communities across platforms. Our results show that most communities exhibit differences in toxicity depending on the platform. We see that toxicity is rooted in the different subcultures as well as in the way in which the platforms operate and their administrators moderate content. However, we note that in general terms Discord is significantly more toxic than Reddit. We offer a detailed analysis of the topics and types of communities in which this happens and why, which will help moderators and policymakers shape their strategies to mitigate the harm on the Web. In particular, we propose practical and effective strategies that Discord can implement to improve its platform moderation.

An Echo Chamber on social media refers to the environment where like-minded people hear the echo of each others' voices, opinions, or beliefs, which reinforce their own. Echo Chambers can turn social media platforms into collaborative venues that polarize and radicalize users rather than broadening their exposure to diverse information. Having a quantified metric for measuring the Echo Chamber effect can aid moderators and policymakers in tracking and mitigating online polarization and radicalization. Existing methods for Echo Chamber detection are either one-dimensional, only considering the network behavior of users while ignoring their semantic behavior, or require demanding supervised labeling, which is both expensive and less generalizable. This paper proposes a new metric to quantify the Echo Chamber effect using Transformer models for context-sensitive processing of natural language (NLP). Our metric quantifies (1) the effect of an Echo Chamber through the inverse effect of user diversity , and (2) polarization by means of user separability between two Echo Chambers in a topic. Leveraging this metric, we further propose an NLP-based embedding that represents the users' activity. Our model is simultaneously effective, computationally cheap, and unsupervised. As our method is unsupervised, it makes existing collaborative moderation efforts to thwart Echo Chamber effects more efficient by addressing the problem of identifying narrow information bases for algorithmic biases and misinformation detection. We run our analysis on three recent highly controversial political topics and a non-controversial topic: Russo-Ukrainian War, Abortion, Gun-Control, and SXSW music festival. Our results offer data-driven findings such as a higher Echo Chamber effect among Republicans over Democrats and diverse explicit support for Ukraine, especially among Democrats. We also observe a direct relationship between the Echo Chamber effect and polarization while observing that the low Echo Chamber effect for the Russo-Ukraine war is accompanied by a low polarization; and vice versa for Gun-Control.

Designing AI systems with the capacity to explain their behaviour is paramount to enable human oversight, facilitate trust, promote acceptance of technology and, ultimately, empower users and improve their experience. There are, however, several challenges to explainable AI, one of which is the generation and selection of explanations from the causal history of a given event. Causal attribution, among other cognitive processes, has been found to be influenced by socio-cultural factors, which suggests that there could be systematic differences in preferences for AI explanations between communities of users according to their cognitive style and socio-cultural characteristics. In this paper, we investigate the relationship between preferences in the explanations provided by belief-desire-intention AI agents, cognitive style (holistic vs analytical), and socio-cultural factors, such as gender, education, social class, and political and religious beliefs. We found a relationship between explanation preference, cognitive style and various socio-cultural characteristics. Holistic cognitive style is associated with preference for goal explanations while analytic cognitive style is associated with preference for belief explanations. Socio-cultural variables that affect explanation preference are gender, religious beliefs, educational attainment, some fields of education, and political party affiliation.

AI assistants such as Alexa, Google Assistant, and Siri, are making their way into the healthcare sector, offering a convenient way for users to access different healthcare services. Trust is a vital factor in the uptake of healthcare services, but the factors affecting trust in voice assistants used for healthcare are under-explored and this specialist domain introduces additional requirements. This study explores the effects of different functional, personal, and risk factors on trust in and adoption of healthcare voice AI assistants (HVAs), generating a partial least squares structural model from a survey of 300 voice assistant users. Our results indicate that trust in HVAs can be significantly explained by functional factors (usefulness, content credibility, quality of service relative to a healthcare professional), together with security, and privacy risks and personal stance in technology. We also discuss differences in terms of trust between HVAs and general-purpose voice assistants as well as implications that are unique to HVAs.

As smart home devices proliferate, protecting the privacy of those who encounter the devices is of the utmost importance both within their own home and in other people's homes. In this study, we conducted a large-scale survey (N=1459) with primary users of and bystanders to smart home devices. While previous work has studied people's privacy experiences and preferences either as smart home primary users or as bystanders, there is a need for a deeper understanding of privacy experiences and preferences in different contexts and across different countries. Instead of classifying people as either primary users or bystanders, we surveyed the same participants across different contexts. We deployed our survey in four countries (Germany, Mexico, the United Kingdom, and the United States) and in two languages (English and Spanish). We found that participants were generally more concerned about devices in their own homes, but perceived video cameras—especially unknown ones—and usability as more concerning in other people's homes. Compared to male participants, female and non-binary participants had less control over configuration of devices and privacy settings—regardless of whether they were the most frequent user. Comparing countries, participants in Mexico were more likely to be comfortable with devices, but also more likely to take privacy precautions around them. We also make cross-contextual recommendations for device designers and policymakers, such as nudges to facilitate social interactions.

When bias mitigation methods are applied to make fairer machine learning models in fairness-related classification settings, there is an assumption that the disadvantaged group should be better off than if no mitigation method was applied. However, this is a potentially dangerous assumption because a “fair” model outcome does not automatically imply a positive impact for a disadvantaged individual—they could still be negatively impacted. Modeling and accounting for those impacts is key to ensure that mitigated models are not unintentionally harming individuals; we investigate if mitigated models can still negatively impact disadvantaged individuals and what conditions affect those impacts in a loan repayment example. Our results show that most mitigated models negatively impact disadvantaged group members in comparison to the unmitigated models. The domain-dependent impacts of model outcomes should help drive future bias mitigation method development.

Since Siri’s release in 2011 there have been a growing number of AI-driven domestic voice assistants that are increasingly being integrated into devices such as smartphones and TVs. But as their presence has expanded, a range of ethical concerns have been identified around the use of voice assistants, such as the privacy implications of having devices that are always listening and the ways that these devices are integrated into the existing social order of the home. This has created a burgeoning area of research across a range of fields including computer science, social science, and psychology. This paper takes stock of the foundations and frontiers of this work through a systematic literature review of 117 papers on ethical concerns with voice assistants. In addition to analysis of nine specific areas of concern, the review measures the distribution of methods and participant demographics across the literature. We show how some concerns, such as privacy, are operationalized to a much greater extent than others like accessibility, and how study participants are overwhelmingly drawn from a small handful of Western nations. In so doing we hope to provide an outline of the rich tapestry of work around these concerns and highlight areas where current research efforts are lacking.

Collaborative cloud platforms make it easier and more convenient for multiple users to work together on files (GoogleDocs, Office365) and store and share them (Dropbox, OneDrive). However, this can lead to privacy and security conflicts between the users involved, for instance when a user adds someone to a shared folder or changes its permissions. Such multiuser conflicts (MCs), though known to happen in the literature, have not yet been studied in-depth. In this paper, we report a study with 1,050 participants about MCs they experienced in the cloud. We show what are the MCs that arise when multiple users work together in the cloud and how and why they arise, what is the prevalence and severity of MCs, what are their consequences on users, and how do users work around MCs. We derive recommendations for designing mechanisms to help users avoid, mitigate, and resolve MCs in the cloud.

To improve user experience, Alexa now allows users to consent to data sharing via voice rather than directing them to the companion smartphone app. While verbal consent mechanisms for voice assistants (VAs) can increase usability, they can also undermine principles core to informed consent. We conducted a Delphi study with experts from academia, industry, and the public sector on requirements for verbal consent in VAs. Candidate requirements were drawn from the literature, regulations, and research ethics guidelines that participants rated based on their relevance to the consent process, actionability by platforms, and usability by end-users, discussing their reasoning as the study progressed. We highlight key areas of (dis)agreement between experts, deriving recommendations for regulators, skill developers, and VA platforms towards crafting meaningful verbal consent mechanisms. Key themes include approaching permissions according to the user’s ability to opt-out, minimising consent decisions, and ensuring platforms follow established consent principles.

The introduction of ChatGPT and the subsequent improvement of Large Language Models (LLMs) have prompted more and more individuals to turn to the use of ChatBots, both for information and assistance with decision-making. However, the information the user is after is often not formulated by these ChatBots objectively enough to be provided with a definite, globally accepted answer.

Personal assistants (PAs) such as Amazon Alexa, Google Assistant and Apple Siri are now widespread. However, without adequate safeguards and controls their use may lead to privacy risks and violations. In this paper, we propose a model for privacy-enhancing PAs. The model is an interpretable AI architecture that combines 1) a dialogue mechanism for understanding the user and getting online feedback from them, with 2) a decision-making mechanism based on case-based reasoning considering both user and scenario similarity. We evaluate our model using real data about users’ privacy preferences, and compare its accuracy and demand for user involvement with both online machine learning and other, more interpretable, AI approaches. Our results show that our proposed architecture is more accurate and requires less intervention from the users than existing approaches.

The General Data Protection Regulation (GDPR) aims to ensure that all personal data processing activities are fair and transparent for the European Union (EU) citizens, regardless of whether these are carried out within the EU or anywhere else. To this end, it sets strict requirements to transfer personal data outside the EU. However, checking these requirements is a daunting task for supervisory authorities, particularly in the mobile app domain due to the huge number of apps available and their dynamic nature. In this paper, we propose a fully automated method for assessing the compliance of Android apps with the GDPR requirements for cross-border personal data transfers. We have applied the method to 4593 apps from the Google Play Store discovering that nearly half of the ones sending personal data are potentially non-compliant with GDPR requirements. These results reveal that there is still a very significant gap between what app providers do in practice and what is intended by the GDPR.

Over the years, most research towards defenses against adversarial attacks on machine learning models has been in the image recognition domain. The ML-based malware detection domain has received less attention despite its importance. Moreover, most work exploring these defenses has focused on several methods but with no strategy when applying them. In this paper, we introduce StratDef, which is a strategic defense system based on a moving target defense approach. We overcome challenges related to the systematic construction, selection, and strategic use of models to maximize adversarial robustness. StratDef dynamically and strategically chooses the best models to increase the uncertainty for the attacker while minimizing critical aspects in the adversarial ML domain, like attack transferability. We provide the first comprehensive evaluation of defenses against adversarial attacks on machine learning for malware detection, where our threat model explores different levels of threat, attacker knowledge, capabilities, and attack intensities. We show that StratDef performs better than other defenses even when facing the peak adversarial threat. We also show that, of the existing defenses, only a few adversarially-trained models provide substantially better protection than just using vanilla models but are still outperformed by StratDef.

Skills, are essential components in Smart Personal Assistants (SPA). The number of skills has grown rapidly, dominated by a changing environment that has no clear business model. Skills can access personal information and this may pose a risk to users. However, there is little information about how this ecosystem works, let alone the tools that can facilitate its study. In this article, we present the largest systematic measurement of the Amazon Alexa skill ecosystem to date. We study developers’ practices in this ecosystem, including how they collect and justify the need for sensitive information, by designing a methodology to identify over-privileged skills with broken privacy policies. We collect 199,295 Alexa skills and uncover that around 43% of the skills (and 50% of the developers) that request these permissions follow bad privacy practices, including (partially) broken data permissions traceability. In order to perform this kind of analysis at scale, we presentSkillVetthat leverages machine learning and natural language processing techniques, and generates high-accuracy prediction sets. We report several concerning practices, including how developers can bypass Alexa's permission system through account linking and conversational skills, and offer recommendations on how to improve transparency, privacy and security. Resulting from the responsible disclosure we did, 13% of the reported issues no longer pose a threat at submission time.

ML models are known to be vulnerable to adversarial query attacks. In these attacks, queries are iteratively perturbed towards a particular class without any knowledge of the target model besides its output. The prevalence of remotely-hosted ML classification models and Machine-Learning-as-a-Service platforms means that query attacks pose a real threat to the security of these systems. To deal with this, stateful defenses have been proposed to detect query attacks and prevent the generation of adversarial examples by monitoring and analyzing the sequence of queries received by the system. Several stateful defenses have been proposed in recent years. However, these defenses rely solely on similarity or out-of-distribution detection methods that may be effective in other domains. In the malware detection domain, the methods to generate adversarial examples are inherently different, and therefore we find that such detection mechanisms are significantly less effective. Hence, in this paper, we present MalProtect, which is a stateful defense against query attacks in the malware detection domain. MalProtect uses several threat indicators to detect attacks. Our results show that it reduces the evasion rate of adversarial query attacks by 80+% in Android and Windows malware, across a range of attacker scenarios. In the first evaluation of its kind, we show that MalProtect outperforms prior stateful defenses, especially under the peak adversarial threat.

Current cloud deletion mechanisms fall short in meeting users’ various deletion needs. They assume all data is deleted the same way—data is temporally removed (or hidden) from users’ cloud accounts before being completely deleted. This assumption neglects users’ desire to have data completely deleted instantly or their preference to have it recoverable for a more extended period. To date, these preferences have not been explored. To address this gap, we conducted a participatory study with four groups of active cloud users (five subjects per group). We examined their deletion preferences and the information they require to aid deletion. In particular, we explored how users want to delete cloud data and identify what information about cloud deletion they consider essential, the time it should be made available to them, and the communication channel that should be used. We show that cloud deletion preferences are complex and multi-dimensional, varying between subjects and groups. Information about deletion should be within reach when needed, for instance, be part of deletion controls. Based on these findings, we discuss the implications of our study in improving the current deletion mechanism to accommodate these preferences.

Language carries implicit human biases, functioning both as a reflection and a perpetuation of stereotypes that people carry with them. Recently, ML-based NLP methods such as word embeddings have been shown to learn such language biases with striking accuracy. This capability of word embeddings has been successfully exploited as a tool to quantify and study human biases. However, previous studies only consider a predefined set of biased concepts to attest (e.g., whether gender is more or less associated with particular jobs), or just discover biased words without helping to understand their meaning at the conceptual level. As such, these approaches can be either unable to find biased concepts that have not been defined in advance, or the biases they find are difficult to interpret and study. This could make existing approaches unsuitable to discover and interpret biases in online communities, as such communities may carry different biases than those in mainstream culture. This paper improves upon, extends, and evaluates our previous data-driven method to automatically discover and help interpret biased concepts encoded in word embeddings. We apply this approach to study the biased concepts present in the language used in online communities and experimentally show the validity and stability of our method.

Smart Personal Assistants (SPAs), such as Amazon Alexa, Google Assistant and Apple Siri, leverage different AI techniques to provide convenient help and assistance to users. However, inappropriate information sharing decisions can lead SPAs to incorrectly disclose user information to undesired parties, or mistakenly block their reasonable access in specific scenarios to desired parties. In fact, reports about privacy violations in SPAs and associated user concerns are well known and understood in the related literature. It is difficult for SPAs to automatically decide how data should be shared with respect to the privacy preferences of the users. We argue norms, which are regarded as shared standards of acceptable behaviour of groups and/or individuals, can be used to govern and reason about the best course of action of SPAs with regards to information sharing, and our work is the first to propose a practical model to address the above issues and govern SPAs based on normative systems and the contextual integrity theory of privacy. We evaluated the performance of the model using a real dataset of user preferences for privacy in SPAs and the results showed a very marked and significant improvement in understanding user preferences and making the right decisions with respect to data sharing.

The unprecedented adoption of messaging platforms for work and recreation has made it an attractive target for malicious actors. In this context, third-party apps (so-called chatbots) offer a variety of attractive functionalities that support the experience in large channels. Unfortunately, under the current permission and deployment models, chatbots in messaging systems could steal information from channels without the victim's awareness. In this paper, we propose a methodology that incorporates static and dynamic analysis for automatically assessing security and privacy issues in messaging platform chatbots. We also provide preliminary findings from the popular Discord platform that highlight the risks that chatbots pose to users. Unlike other popular platforms like Slack or MS Teams, Discord does not implement user-permission checks---a task entrusted to third-party developers. Among others, we find that 55% of chatbots from a leading Discord repository request the "administrator" permission, and only 4.35% of chatbots with permissions actually provide a privacy policy.

Smart Voice Assistants are transforming the way users interact with technology. This transformation is mostly fostered by the proliferation of voice-driven applications (called skills) offered by third-party developers through an online market. We see how the number of skills has rocked in recent years, with the Amazon Alexa skill ecosystem growing from just 135 skills in early 2016 to about 125k skills in early 2021. Along with the growth in skills, there is increasing concern over the risks that third-party skills pose to users’ privacy. In this paper, we perform a systematic and longitudinal measurement study of the Alexa marketplace. We shed light on how this ecosystem evolves using data collected across three years between 2019 and 2021. We demystify developers’ data disclosure practices and present an overview of the third-party ecosystem. We see how the research community continuously contribute to the market’s sanitation, but the Amazon vetting process still requires significant improvement. We perform a responsible disclosure process reporting 675 skills with privacy issues to both Amazon and all affected developers, out of which 246 skills suffer from important issues (i.e., broken traceability). We see that 107 out of the 246 (43.5%) skills continue to display broken traceability almost one year after being reported. As a result, the overall state of affairs has improved in the ecosystem over the years. Yet, newly submitted skills and unresolved known issues pose an endemic risk.

Abstract Multiuser Privacy (MP) concerns the protection of personal information in situations where such information is co-owned by multiple users. MP is particularly problematic in collaborative platforms such as online social networks (OSN). In fact, too often OSN users experience privacy violations due to conflicts generated by other users sharing content that involves them without their permission. Previous studies show that in most cases MP conflicts could be avoided, and are mainly due to the difficulty for the uploader to select appropriate sharing policies. For this reason, we present ELVIRA, the first fully explainable personal assistant that collaborates with other ELVIRA agents to identify the optimal sharing policy for a collectively owned content. An extensive evaluation of this agent through software simulations and two user studies suggests that ELVIRA, thanks to its properties of being role-agnostic, adaptive, explainable and both utility- and value-driven, would be more successful at supporting MP than other approaches presented in the literature in terms of (i) trade-off between generated utility and promotion of moral values, and (ii) users’ satisfaction of the explained recommended output.

This paper reports on empirical work conducted to study perceptions of unfair treatment caused by automated computational systems. While the pervasiveness of algorithmic bias has been widely acknowledged, and perceptions of fairness are commonly studied in Human Computer Interaction, there is a lack of research on how unfair treatment by automated computational systems is experienced by users from disadvantaged and marginalised backgrounds. There is a need for more diversification in terms of the investigated users, domains, and tasks, and regarding the strategies that users employ to reduce harm. To unpack these issues, we ran a prescreened survey of 663 participants, oversampling those with at-risk characteristics. We collected occurrences and types of conflicts regarding unfair and discriminatory treatment and systems, as well as the actions taken towards resolving these situations. Drawing on intersectional research, we combine qualitative and quantitative approaches in order to highlight the nuances around power and privilege in the perceptions of automated computational systems. Among our participants, we discuss experiences of computational essentialism, attribute-based exclusion, and expected harm. We derive suggestions to address these perceptions of unfairness as they occur.

Smart Home Personal Assistants (SPA) have a complex ecosystem that enables them to carry out various tasks on behalf of the user with just voice commands. SPA capabilities are continually growing, with over a hundred thousand third-party skills in Amazon Alexa, covering several categories, from tasks within the home (e.g. managing smart devices) to tasks beyond the boundaries of the home (e.g. purchasing online, booking a ride). In the SPA ecosystem, information flows through several entities including SPA providers, third-party skills providers, providers of Smart Devices, other users and external parties. Prior studies have not explored privacy norms in the SPA ecosystem, i.e., the acceptability of these information flows. In this paper, we study privacy norms in SPAs based on Contextual Integrity through a large-scale study with 1,738 participants. We also study the influence that the Contextual Integrity parameters and personal factors have on the privacy norms. Further, we identify the similarities in terms of the Contextual Integrity parameters of the privacy norms studied to distill more general privacy norms, which could be useful, for instance, to establish suitable privacy defaults in SPA. We finally provide recommendations for SPA and third-party skill providers based on the privacy norms studied.

Individuals are known to lie and/or provide untruthful data when providing information online as a way to protect their privacy. Prior studies have attempted to explain when and why individuals lie online. However, no work has examined into how people lie or provide untruthful data online, i.e. the specific strategies they follow to provide untruthful data, or attempted to predict whether people would be truthful or not depending on the specific question/data. To close this gap, we present a large-scale study with over 800 participants. Based on it, we show that it is possible to predict whether users are truthful or not using machine learning with very high accuracy (89.7%). We also identify four main strategies people employ to provide untruthful data and show the factors that influence the choices of their strategies. We discuss the implications of findings and argue that understanding privacy lies at this level can help both users and data collectors.

Individuals share increasing amounts of personal multimedia data, exposing themselves (uploaders) as well as others (data subjects). Non-consensual sharing of multimedia data that depicts others raises so-called multiparty privacy conflicts (MPCs), which can have severe consequences. To limit the incidence of MPCs, a family of Precautionary mechanisms have recently been developed that force uploaders to collaborate with the other data subjects to prevent MPCs. However, there is still very little work on understanding how users perceive the precautionary mechanisms together with which ones they prefer and why. In addition, precautionary mechanisms have some limitations, e.g., they require linking content to the co-owners' identity. Therefore, we also explore alternatives to precautionary mechanisms and propose a new class of solutions-Dissuasive mechanisms-that aim at deterring the uploaders from sharing without consent. We then present a user-centric comparison of precautionary and dissuasive mechanisms, through a large-scale survey (N = 1792; a representative sample of adult Internet users). Our results showed that respondents prefer precautionary to dissuasive mechanisms. These enforce collaboration, provide more control to the data subjects, but also they reduce uploaders' uncertainty around what is considered appropriate for sharing. We learned that threatening legal consequences is the most desirable dissuasive mechanism, and that respondents prefer the mechanisms that threaten users with immediate consequences (compared with delayed consequences). Dissuasive mechanisms are in fact well received by frequent sharers and older users, while precautionary mechanisms are preferred by women and younger users. We discuss the implications for design, including considerations about side leakages, consent collection, and censorship.

editorial Free Access Share on Guest Editors’ Introduction Authors: Özgür Kafali University of Kent, Canterbury, Kent, UK University of Kent, Canterbury, Kent, UKView Profile , Natalia Criado King’s College London, London, UK King’s College London, London, UKView Profile , Martin Rehak Cisco Systems, Czech Republic Cisco Systems, Czech RepublicView Profile , Jose M. Such King’s College London, London, UK King’s College London, London, UKView Profile , Pinar Yolum Utrecht University, Utrecht, Netherlands Utrecht University, Utrecht, NetherlandsView Profile Authors Info & Claims ACM Transactions on Internet TechnologyVolume 18Issue 3August 2018 Article No.: 26pp 1–4https://doi.org/10.1145/3177884Published:06 March 2018Publication History 0citation462DownloadsMetricsTotal Citations0Total Downloads462Last 12 Months21Last 6 weeks4 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

The problems that social media users have in appropriately controlling access to their content has been well documented in previous research. A promising method of providing assistance to users is by learning from the access control decisions made by them and making future recommendations. In this paper, we present REACT, a learning mechanism which utilizes information available in the social network in conjunction with information about the content to be shared to provide users with access control recommendations. We demonstrate the highly accurate performance of REACT through a detailed empirical evaluation and also discuss ways of personalizing it for different users in order to improve performance even further.

Social Media Aggregator (SMA) applications present a platform enabling users to manage multiple Social Networking Sites (SNS) in one convenient application, which results in a unique concentration of data from several SNS accounts in addition to the user's mobile phone data available to them. In this paper, we provide a detailed privacy assessment of 13 popular SMAs from 3 app stores by using a three-step methodology by inspecting the mobile data and social media data accessed by these applications, checking for privacy policies and their compliance with distributors' vetting policies and performing a qualitative assessment of traceability between privacy policies and the actual transparency and control mechanisms offered to users by the apps' interfaces. Our results demonstrate a variation in data accessed by the individual applications, an absence of privacy policies for 5 of the SMAs evaluated, and a lack of traceability between privacy policies and transparency and control of interface operations.

Items in social media such as photos may be co-owned by multiple users, i.e., the sharing decisions of the ones who upload them have the potential to harm the privacy of the others. Previous works uncovered coping strategies by co-owners to manage their privacy, but mainly focused on general practices and experiences. We establish an empirical base for the prevalence, context and severity of privacy conflicts over co-owned photos. To this aim, a parallel survey of pre-screened 496 uploaders and 537 co-owners collected occurrences and type of conflicts over co-owned photos, and any actions taken towards resolving them. We uncover nuances and complexities not known before, including co-ownership types, and divergences in the assessment of photo audiences. We also find that an all-or-nothing approach seems to dominate conflict resolution, even when parties actually interact and talk about the conflict. Finally, we derive key insights for designing systems to mitigate these divergences and facilitate consensus.

We discuss the problem of privacy in autonomous systems, introducing different conceptualizations and perspectives on privacy to assess the threats that autonomous systems may pose to privacy. After this, we outline socio-technical and legal measures that should be put in place to mitigate these threats. Beyond privacy threats and countermeasures, we also argue how autonomous systems may be, at the same time, the key to address some of the most challenging and pressing privacy problems nowadays and in the near future.

In the context of using norms for controlling multiagent systems, a vitally important question that has not yet been addressed in the literature is the development of mechanisms for monitoring norm compliance under partial action observability. This paper proposes the reconstruction of unobserved actions to tackle this problem. In particular, we formalize the problem of reconstructing unobserved actions, and propose an information model and algorithms for monitoring norms under partial action observability using two different processes for reconstructing unobserved actions. Our evaluation shows that reconstructing unobserved actions increases significantly the number of norm violations and fulfilments detected.

Social network services (SNSs) enable users to conveniently share personal information. Often, the information shared concerns other people, especially other members of the SNS. In such situations, two or more people can have conflicting privacy preferences; thus, an appropriate sharing policy may not be apparent. We identify such situations as multiuser privacy scenarios . Current approaches propose finding a sharing policy through preference aggregation. However, studies suggest that users feel more confident in their decisions regarding sharing when they know the reasons behind each other’s preferences. The goals of this paper are (1) understanding how people decide the appropriate sharing policy in multiuser scenarios where arguments are employed, and (2) developing a computational model to predict an appropriate sharing policy for a given scenario. We report on a study that involved a survey of 988 Amazon Mechanical Turk (MTurk) users about a variety of multiuser scenarios and the optimal sharing policy for each scenario. Our evaluation of the participants’ responses reveals that contextual factors, user preferences, and arguments influence the optimal sharing policy in a multiuser scenario. We develop and evaluate an inference model that predicts the optimal sharing policy given the three types of features. We analyze the predictions of our inference model to uncover potential scenario types that lead to incorrect predictions, and to enhance our understanding of when multiuser scenarios are more or less prone to dispute.

Social media users often find it difficult to make appropriate access control decisions which govern how they share their information with a potentially large audience on these platforms. Community detection algorithms have been previously put forth as a solution which can help users by automatically partitioning their friend network. These partitions can then be used by the user as a basis for making access control decisions. Previous works which leverage communities for enhancing access control mechanisms assume that members of the same community will have the same access to a user's content, but whether or to what extent this assumption is correct is a lingering question. In this paper, we empirically evaluate a goodness of fit between the communities created by implementing 8 community detection algorithms on the friend networks of users and the access control decisions made by them during a user study. We also analyze whether personal characteristics of the users or the nature of the content play a role in the performance of the algorithms. The results indicate that community detection algorithms may be useful for creating default access control policies for users who exhibit a relatively more static access control behaviour. For users showing great variation in their access control decisions across the board (both in terms of number and actual members), we found that community detection algorithms performed poorly.

Social media involve many shared items, such as photos, which may concern more than one user. The challenge is that users’ individual privacy preferences for the same item may conflict, so an approach that simply merges in some way the users’ privacy preferences may provide unsatisfactory results. Previous proposals to deal with the problem were either time-consuming or did not consider compromises to solve these conflicts (e.g., by considering unilaterally imposed approaches only). We propose a negotiation mechanism for users to agree on a compromise for these conflicts. The second challenge we address in this article relates to the exponential complexity of such a negotiation mechanism. To address this, we propose heuristics that reduce the complexity of the negotiation mechanism and show how substantial benefits can be derived from the use of these heuristics through extensive experimental evaluation that compares the performance of the negotiation mechanism with and without these heuristics. Moreover, we show that one such heuristic makes the negotiation mechanism produce results fast enough to be used in actual social media infrastructures with near-optimal results.

Items shared through Social Media may affect more than one user's privacy-e.g., photos that depict multiple users, comments that mention multiple users, events in which multiple users are invited, etc. The lack of multi-party privacy management support in current mainstream Social Media infrastructures makes users unable to appropriately control to whom these items are actually shared or not. Computational mechanisms that are able to merge the privacy preferences of multiple users into a single policy for an item can help solve this problem. However, merging multiple users' privacy preferences is not an easy task, because privacy preferences may conflict, so methods to resolve conflicts are needed. Moreover, these methods need to consider how users' would actually reach an agreement about a solution to the conflict in order to propose solutions that can be acceptable by all of the users affected by the item to be shared. Current approaches are either too demanding or only consider fixed ways of aggregating privacy preferences. In this paper, we propose the first computational mechanism to resolve conflicts for multi-party privacy management in Social Media that is able to adapt to different situations by modelling the concessions that users make to reach a solution to the conflicts. We also present results of a user study in which our proposed mechanism outperformed other existing approaches in terms of how many times each approach matched users' behaviour.

Social networking services (SNSs) such as Facebook and Twitter have experienced explosive growth during the few past years. Millions of users have created their profiles on these services because they experience great benefits in terms of friendship. SNSs can help people to maintain their friendships, organize their social lives, start new friendships, and meet others who share their hobbies and interests. However, all these benefits can be eclipsed by the privacy hazards that affect people in SNSs. People expose intimate information about their lives on SNSs, and this information affects the way others think about them. It is crucial that users be able to control how their information is distributed through the SNSs and decide who can access it. This article presents a list of privacy threats that can affect SNS users and what requirements privacy mechanisms should fulfill to prevent this threats. Then the article reviews current approaches and analyzes to what extent they cover the requirements.

Abstract Security is becoming a major concern in multiagent systems, since an agent's incorrect or inappropriate behaviour may cause non‐desired effects, such as money and data loss. Some multiagent platforms (MAP) are now providing baseline security features, such as authentication, authorization, integrity and confidentiality. However, they fail to support other features related to the sociability skills of agents such as agent groups. What is more, none of the listed MAPs provide a mechanism for preserving the privacy of the users (regarding their identities) that run their agents on such MAPs. In this paper, we present the security infrastructure (SI) of the Magentix MAP, which supports agent groups and preserves user identity privacy. The SI is based on identities that are assigned to all the different entities found in Magentix (users, agents and agent groups). We also provide an evaluation of the SI describing an example application built on top of Magentix and a performance evaluation of it. Copyright © 2010 John Wiley & Sons, Ltd.