As AI agents attempt to autonomously act on users' behalf, they raise transparency and control issues. We argue that permission-based access control is indispensable in providing meaningful control to the users, but conventional permission models are inadequate for the automated agentic execution paradigm. We therefore propose automated permission management for AI agents. Our key idea is to conduct a user study to identify the factors influencing users' permission decisions and to encode these factors into an ML-based permission management assistant capable of predicting users' future decisions. We find that participants' permission decisions are influenced by communication context but importantly individual preferences tend to remain consistent within contexts, and align with those of other participants. Leveraging these insights, we develop a permission prediction model achieving 85.1% accuracy overall and 94.4% for high-confidence predictions. We find that even without using permission history, our model achieves an accuracy of 66.9%, and a slight increase of training samples (i.e., 1-4) can substantially increase the accuracy by 10.8%.

LLM app (tool) ecosystems are rapidly evolving to support sophisticated use cases that often require extensive user data collection. Given that LLM apps are developed by third parties and anecdotal evidence indicating inconsistent enforcement of policies by LLM platforms, sharing user data with these apps presents significant privacy risks. In this paper, we aim to bring transparency in data practices of LLM app ecosystems. We examine OpenAI's GPT app ecosystem as a case study. We propose an LLM-based framework to analyze the natural language specifications of GPT Actions (custom tools) and assess their data collection practices. Our analysis reveals that Actions collect excessive data across 24 categories and 145 data types, with third-party Actions collecting 6.03% more data on average. We find that several Actions violate OpenAI's policies by collecting sensitive information, such as passwords, which is explicitly prohibited by OpenAI. Lastly, we develop an LLM-based privacy policy analysis framework to automatically check the consistency of data collection by Actions with disclosures in their privacy policies. Our measurements indicate that the disclosures for most of the collected data types are omitted, with only 5.8% of Actions clearly disclosing their data collection practices.

Explainability is increasingly recognized as an enabling technology for the broader adoption of machine learning (ML), particularly for safety-critical applications. This has given rise to explainable ML, which seeks to enhance the explainability of neural networks through the use of explanators. Yet, the pursuit for better explainability inadvertently leads to increased security and privacy risks. While there has been considerable research into the security risks of explainable ML, its potential privacy risks remain under-explored.To bridge this gap, we present a systematic study of privacy risks in explainable ML through the lens of membership inference. Building on the observation that, besides the accuracy of the model, robustness also exhibits observable differences among member samples and non-member samples, we develop a new membership inference attack. This attack extracts additional membership features from changes in model confidence under different levels of perturbations guided by the importance highlighted by the attribution maps in the explanators. Intuitively, perturbing important features generally results in a bigger loss in confidence for member samples. Using the member-non-member differences in both model performance and robustness, an attack model is trained to distinguish the membership. We evaluated our approach with seven popular explanators across various benchmark models and datasets. Our attack demonstrates there is non-trivial privacy leakage in current explainable ML methods. Furthermore, such leakage issue persists even if the attacker lacks the knowledge of training datasets or target model architectures. Lastly, we also found existing model and output-based defense mechanisms are not effective in mitigating this new attack.

LiDAR-based perception is a central component of autonomous driving, playing a key role in tasks such as vehicle localization and obstacle detection. Since the safety of LiDAR-based perceptual pipelines is critical to safe autonomous driving, a number of past efforts have investigated its vulnerability under adversarial perturbations of raw point cloud inputs. However, most such efforts have focused on investigating the impact of such perturbations on predictions (integrity), and little has been done to understand the impact on latency (availability), a critical concern for real-time cyber-physical systems. We present the first systematic investigation of the availability of LiDAR detection pipelines, and SlowLiDAR, an adversarial perturbation attack that maximizes LiDAR detection runtime. The attack overcomes the technical challenges posed by the non-differentiable parts of the LiDAR detection pipelines by using differentiable proxies and uses a novel loss function that effectively captures the impact of adversarial perturbations on the execution time of the pipeline. Extensive experimental results show that SlowLiDAR can significantly increase the latency of the six most popular LiDAR detection pipelines while maintaining imperceptibility11Code is available at: https://github.com/WUSTL-CSPL/SlowLiDAR.

The field of text-to-image generation has made remarkable strides in creating high-fidelity and photorealistic images. As this technology gains popularity, there is a growing concern about its potential security risks. However, there has been limited exploration into the robustness of these models from an adversarial perspective. Existing research has primarily focused on untargeted settings, and lacks holistic consideration for reliability (attack success rate) and stealthiness (imperceptibility). In this paper, we propose RIATIG, a reliable and imperceptible adversarial attack against text-to-image models via inconspicuous examples. By formulating the example crafting as an optimization process and solving it using a genetic-based method, our proposed attack can generate imperceptible prompts for text-to-image generation models in a reliable way. Evaluation of six popular text-to-image generation models demonstrates the efficiency and stealthiness of our attack in both white-box and black-box settings. To allow the community to build on top of our findings, we've made the artifacts available11Code is available at: https://github.com/WUSTL-CSPL/RIATIG.

Tiny machine learning (TinyML) is an essential component of emerging smart microcontrollers (MCUs). However, the protection of the intellectual property (IP) of the model is an increasing concern due to the lack of desktop/server-grade resources on these power-constrained devices. In this paper, we propose STML, a system and algorithm co-design to Secure IP of TinyML on MCUs with ARM TrustZone. Our design jointly optimizes memory utilization and latency while ensuring the security and accuracy of emerging models. We implemented a prototype and benchmarked with 7 models, demonstrating STML reduces 40% of model protection runtime overhead on average.

Recent advances in large language models (LMs) have facilitated their ability to synthesize programming code. However, they have also raised concerns about intellectual property (IP) rights violations. Despite the significance of this issue, it has been relatively less explored. In this paper, we aim to bridge the gap by presenting CodeIPPrompt, a platform for automatic evaluation of the extent to which code language models may reproduce licensed programs. It comprises two key components: prompts constructed from a licensed code database to elicit LMs to generate IP-violating code, and a measurement tool to evaluate the extent of IP violation of code LMs. We conducted an extensive evaluation of existing open-source code LMs and commercial products and revealed the prevalence of IP violations in all these models. We further identified that the root cause is the substantial proportion of training corpus subject to restrictive licenses, resulting from both intentional inclusion and inconsistent license practice in the real world. To address this issue, we also explored potential mitigation strategies, including fine-tuning and dynamic token filtering. Our study provides a testbed for evaluating the IP violation issues of the existing code generation platforms and stresses the need for a better mitigation strategy.

The proliferation of real-time cyber-physical systems (CPS) is making profound changes to our daily life. Many real-time CPSs are security and safety-critical because of their continuous interactions with the physical world. While the general perception is that the security protection mechanism deployment is often absent in real-time embedded systems, there is no existing empirical study that measures the adoption of these mechanisms in the ecosystem. To bridge this gap, we conduct a measurement study for real-time embedded firmware from both a security perspective and a real-time perspective. To begin with, we collected more than 16 terabytes of embedded firmware and sampled 1,000 of them for the study. Then, we analyzed the adoption of security protection mechanisms and their potential impacts on the timeliness of real-time embedded systems. Besides, we measured the scheduling algorithms supported by real-time embedded systems since they are also security-critical.

The rapid convergence of legacy industrial infrastructures with intelligent networking and computing technologies (e.g., 5G, software-defined networking, and artificial intelligence), have dramatically increased the attack surface of industrial cyber-physical systems (CPSs). However, withstanding cyber threats to such large-scale, complex, and heterogeneous industrial CPSs has been extremely challenging, due to the insufficiency of high-quality attack examples. In this article, we propose a novel federated deep learning scheme, named DeepFed, to detect cyber threats against industrial CPSs. Specifically, we first design a new deep learning-based intrusion detection model for industrial CPSs, by making use of a convolutional neural network and a gated recurrent unit. Second, we develop a federated learning framework, allowing multiple industrial CPSs to collectively build a comprehensive intrusion detection model in a privacy-preserving way. Further, a Paillier cryptosystem-based secure communication protocol is crafted to preserve the security and privacy of model parameters through the training process. Extensive experiments on a real industrial CPS dataset demonstrate the high effectiveness of the proposed DeepFed scheme in detecting various types of cyber threats to industrial CPSs and the superiorities over state-of-the-art schemes.

The Internet of energy (IoE), envisioned to be a promising paradigm of the Internet of things (IoT), is characterized by the deep integration of various distributed energy systems. However, the fusion of heterogeneous IoE communication networks creates a new threat landscape. To thwart and mitigate various types of cyber threats to IoE networks, this paper proposes a novel intrusion detection system (IDS) based on a designed residual network with attention long short term memory (ReAL). Specifically, we design a light gradient boosting machine (LightGBM)-based feature selection method to identify the most useful features. Then, a residual network (ResNet) and a long short term memory neural network with an attention mechanism (ALSTM) are employed, to extract temporal patterns of network traffic events. After that, these patterns are orchestrated to identify the anomalies in IoE networks. The high effectiveness of the proposed IDS is validated on a real IoE dataset.