Large Vision-Language Models (LVLMs) are vulnerable to a growing array of multimodal jailbreak attacks, necessitating defenses that are both generalizable to novel threats and efficient for practical deployment.Many current strategies fall short, either targeting specific attack patterns, which limits generalization, or imposing high computational overhead.While lightweight anomaly-detection methods offer a promising direction, we find that their common one-class design tends to confuse unseen benign inputs with malicious ones, leading to unreliable over-rejection.To address this, we propose Representational Contrastive Scoring (RCS), a framework built on a key insight: the most potent safety signals reside within the LVLM's own internal representations.Our approach inspects the internal geometry of these representations, learning a lightweight projection to maximally separate benign and malicious inputs in safety-critical layers.This enables a simple yet powerful contrastive score that differentiates true malicious intent from mere distribution shift.Our instantiations, MCD (Mahalanobis Contrastive Detection) and KCD (K-nearest Contrastive Detection), achieve state-of-the-art performance on a challenging evaluation protocol designed to test generalization to unseen attack types.This work demonstrates that effective jailbreak detection can be achieved by applying simple, interpretable statistical methods to the internal representations, offering a practical path towards safer LVLM deployment.Our code is available on Github 1 .

Knowledge distillation is a widely adopted technique for transferring capabilities from LLMs to smaller, more efficient student models.However, unauthorized use of knowledge distillation takes unfair advantage of the considerable effort and cost put into developing frontier models.We investigate methods for modifying teacher-generated reasoning traces to achieve two objectives that deter unauthorized distillation: (1) anti-distillation, or degrading the training usefulness of query responses, and (2) API watermarking, which embeds verifiable signatures in student models.We introduce several approaches for dynamically rewriting a teacher's reasoning outputs while preserving answer correctness and semantic coherence.Two of these leverage the rewriting capabilities of LLMs, while others use gradient-based techniques.Our experiments show that a simple instruction-based rewriting approach achieves a strong anti-distillation effect while maintaining or even improving teacher performance.Furthermore, we show that our rewriting approach also enables embedding watermarks that can be reliably detected with essentially no false alarms.Our code is available at https:// github.com/xhOwenMa/trace-rewriting.

As AI agents attempt to autonomously act on users' behalf, they raise transparency and control issues. We argue that permission-based access control is indispensable in providing meaningful control to the users, but conventional permission models are inadequate for the automated agentic execution paradigm. We therefore propose automated permission management for AI agents. Our key idea is to conduct a user study to identify the factors influencing users' permission decisions and to encode these factors into an ML-based permission management assistant capable of predicting users' future decisions. We find that participants' permission decisions are influenced by communication context but importantly individual preferences tend to remain consistent within contexts, and align with those of other participants. Leveraging these insights, we develop a permission prediction model achieving 85.1% accuracy overall and 94.4% for high-confidence predictions. We find that even without using permission history, our model achieves an accuracy of 66.9%, and a slight increase of training samples (i.e., 1-4) can substantially increase the accuracy by 10.8%.

Electric vehicles (EVs) have become one of the promising solutions to the ever-evolving environmental and energy crisis. The key to the wide adoption of EVs is a pervasive charging infrastructure, composed of both the private/home chargers and the public/commercial charging stations. The security of EV charging, however, has not been thoroughly investigated. This paper investigates the communication mechanisms between the chargers and EVs, and exposes the lack of protection on the authenticity in the SAE J1772 charging control protocol. To showcase our discoveries, we propose a new class of attacks, ChargeX, which aims to manipulate the charging states or charging rates of EV chargers with the goal of disrupting the charging schedules, causing denial of service (DoS), or degrading the battery performance. ChargeX inserts a hardware attack circuit to strategically modify the charging control signals. We design and implement multiple attack systems, and evaluate the attacks on a public charging station and two home chargers using a simulated vehicle load in the lab environment. Extensive experiments on different types of chargers demonstrate the effectiveness and generalization of ChargeX. Specifically, we demonstrate that ChargeX can force a Tesla’s charging state to switch from “stand by” to “charging”, potentially leading to overcharging. Additionally, ChargeX can transition any charging state to an error state, effectively launching a DoS attack on Tesla. If deployed, ChargeX may significantly demolish people’s trust in the EV charging infrastructure.

Prompt injection attacks pose a critical threat to large language models (LLMs), enabling goal hijacking and data leakage.Prompt guard models, though effective in defense, suffer from over-defense-falsely flagging benign inputs as malicious due to trigger word bias.To address this issue, we introduce NotInject, an evaluation dataset that systematically measures over-defense across various prompt guard models.NotInject contains 339 benign samples enriched with trigger words common in prompt injection attacks, enabling fine-grained evaluation.Our results show that state-of-the-art models suffer from over-defense issues, with accuracy dropping close to random guessing levels (60%).To mitigate this, we propose PIGuard, a novel prompt guard model that incorporates a new training strategy, Mitigating Over-defense for Free (MOF), which significantly reduces the bias on trigger words.PIGuard demonstrates state-of-the-art performance on diverse benchmarks including Not-Inject, surpassing the existing best model by 30.4%, offering a robust and open-source solution for detecting prompt injection attacks.The code and datasets are released at https: //github.com/leolee99/PIGuard.

With the proliferation of Cyber-Physical Systems (CPSs) in daily life, the security of these systems is becoming an pressing problem. Fuzz testing has recently gained attention as a promising approach for automatically detecting vulnerabilities, however, the prohibitively large search space of physical and cyber inputs remains an open research challenge. To address this gap, the paper draws on control theory, leveraging physics-informed control models to guide exploration of the input space. We design and develop ConTest, a fuzzing tool that leverages Lyapunov functions of the control model for both detection and mutation to efficiently search through the parameter space with a provable guarantee on the effectiveness of bug-finding effectiveness under bounded dynamic model errors. We implemented a prototype of ConTest and deployed it to detect spatial and temporal input validation bugs in two representative robotic vehicle (RV) platforms, ArduPilot and PX4. A total of 253 input validation bugs were found, 58 of them being zero-day bugs, and 54 of them were acknowledged by the vendors.

Federated learning enables decentralized model training while preserving data privacy. However, since the learning process overlays the physical network infrastructure, the efficiency of learning can be impacted by network connectivity. In this work, we conducted extensive experiments to empirically characterize the impacts and leverage the insights to propose an adaptive federation framework, where clients with limited bandwidth are only prompted to transmit adaptively compressed gradient updates when the gradient similarity score is similar between the local and global models. Our evaluation in simulated environments and on real hardware devices shows bandwidth savings of 60% to 78% compared to state-of-the-art methods.

Han Liu, Ruoyao Wen, Srijith Nair, Jia Liu, Wenjing Lou, Chongjie Zhang, William Yeoh, Yevgeniy Vorobeychik, Ning Zhang. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

LLM app (tool) ecosystems are rapidly evolving to support sophisticated use cases that often require extensive user data collection. Given that LLM apps are developed by third parties and anecdotal evidence indicating inconsistent enforcement of policies by LLM platforms, sharing user data with these apps presents significant privacy risks. In this paper, we aim to bring transparency in data practices of LLM app ecosystems. We examine OpenAI's GPT app ecosystem as a case study. We propose an LLM-based framework to analyze the natural language specifications of GPT Actions (custom tools) and assess their data collection practices. Our analysis reveals that Actions collect excessive data across 24 categories and 145 data types, with third-party Actions collecting 6.03% more data on average. We find that several Actions violate OpenAI's policies by collecting sensitive information, such as passwords, which is explicitly prohibited by OpenAI. Lastly, we develop an LLM-based privacy policy analysis framework to automatically check the consistency of data collection by Actions with disclosures in their privacy policies. Our measurements indicate that the disclosures for most of the collected data types are omitted, with only 5.8% of Actions clearly disclosing their data collection practices.

In safety critical computing, real-time and security concerns are often considered separately, though the behavior of a scheduling model itself may be an attack surface which can be exploited by an attacker to reduce system performance. In this work, we explore how the semantics of mode changes in mixed-criticality systems could be used as one such attack vector. This attack, dubbed Mad Monk, uses a mixed criticality scheduler's mode switches against itself by allowing a task of a lower criticality to interfere with tasks of a higher criticality, thereby forcing a disruptive mode switch which could possibly reduce service to some tasks. We describe this attack in detail, along with a case study demonstrating its risk. Furthermore, extensive simulations of this attack demonstrate its potential effectiveness based on a variety of timing and system factors.

Large Language Models (LLMs) are increasingly central to agentic systems due to their strong reasoning and planning capabilities. By interacting with external environments through predefined tools, these agents can carry out complex user tasks. Nonetheless, this interaction also introduces the risk of prompt injection attacks, where malicious inputs from external sources can mislead the agent’s behavior, potentially resulting in economic loss, privacy leakage, or system compromise. System-level defenses have recently shown promise by enforcing static or predefined policies, but they still face two key challenges: the ability to dynamically update security rules and the need for memory stream isolation. To address these challenges, we propose DRIFT, a Dynamic Rule-based Isolation Framework for Trustworthy agentic systems, which enforces both control- and data-level constraints. A Secure Planner first constructs a minimal function trajectory and a JSON-schema-style parameter checklist for each function node based on the user query. A Dynamic Validator then monitors deviations from the original plan, assessing whether changes comply with privilege limitations and the user's intent. Finally, an \textit{Injection Isolator} detects and masks any instructions that may conflict with the user query from the memory stream to mitigate long-term risks. We empirically validate the effectiveness of DRIFT on the AgentDojo and ASB benchmark, demonstrating its strong security performance while maintaining high utility across diverse models—showcasing both its robustness and adaptability. The code is released at https://github.com/SaFoLab-WISC/DRIFT.

Self-suspension behavior happens when a job has to wait for some activity to complete and results in substantial schedulability degradation in real-time systems. Despite extensive studies for self-suspending real-time task systems, the state of the art has barely addressed the optimality of the scheduling algorithms, especially for tasks with dynamic self-suspension. In this paper, we explore optimal priority assignment for periodic real-time tasks with dynamic self-suspension under Task-level Fixed-Priority (T-FP) scheduling. To that end, we provide exact schedulability tests for frame-based and synchronous harmonic tasks. We show that the Suspension-Aware Deadline-Monotonic (SADM) priority assignment is an optimal fixed-priority scheduler for many scenarios. Further, for cases where SADM is not optimal, we adopt Audsley's Optimal Priority Assignment (OPA) approach to derive an optimal fixedpriority assignment. Evaluation results show that the exact tests outperform state-of-the-art schedulability tests from the literature, and that optimal priority assignments significantly improve schedulability over classical priority assignments.

Learning reward models from pairwise comparisons is a fundamental component in a number of domains, in-cluding autonomous control, conversational agents, and rec-ommendation systems, as part of a broad goal of aligning automated decisions with user preferences. These approaches entail collecting preference information from people, with feedback often provided anonymously. Since preferences are subjective, there is no gold standard to compare against; yet, reliance of high-impact systems on preference learning creates a strong motivation for malicious actors to skew data collected in this fashion to their ends. We investigate the nature and extent of this vulnerability by considering an attacker who can flip a small subset of preference comparisons to either promote or demote a target outcome. We propose two classes of algorithmic approaches for these attacks: a gradient-based framework, and several variants of rank-by-distance methods. Next, we evaluate the efficacy of best attacks in both these classes in successfully achieving malicious goals on datasets from three domains: autonomous control, recommendation system, and textual prompt-response preference learning. We find that the best attacks are often highly successful, achieving in the most extreme case 100% success rate with only 0.3% of the data poisoned. However, which attack is best can vary significantly across domains. In addition, we observe that the simpler and more scalable rank-by-distance approaches are often competitive with, and on occasion significantly outper-form, gradient-based methods. Finally, we show that state-of-the-art defenses against other classes of poisoning attacks exhibit limited efficacy in our setting.

No abstract available.

Android system provides application developers with the ability to define custom permissions, which serve to regulate the sharing of resources and functionalities with other applications. However, developers' improper development practices can render the permission mechanism ineffective, facilitating easy exploitation by attackers. This paper presents a comprehensive examination of the problematic practices surrounding custom permissions employed by developers, referred to as Bad Practices of Custom Permissions (BPCP issues). To accomplish this, we conducted an empirical study and identified nine common BPCP issue patterns that can lead to various adverse consequences, such as installation failures, crashes, and even component hijacking. To automatically identify these patterns of bad practices, we devised PERMEAGRE, a static analysis tool. Employing PERMEAGRE, we performed a large-scale analysis of 83,085 applications obtained from seven major app markets, aiming to detect instances of BPCP issues. The results revealed that more than 26% of the analyzed apps contained at least one issue, and a significant number of apps had garnered millions of downloads. Our analysis delved into the underlying causes of these issues. Consequently, this analysis sheds light on the potential threat landscape associated with bad practices in custom permissions, emphasizing the urgent requirement for effective mitigation strategies.

Autonomous vehicles (AVs) empowered by deep neural networks (DNNs) are bringing transformative changes to our society. However, they are generally susceptible to adversarial attacks, especially physically realizable perturbations that can mislead perception and cause catastrophic outcomes. While existing defenses have shown success, there remains a pressing need for improved robustness while maintaining efficiency to meet real-time system operations.

The surge in volumes of video data offers unprecedented opportunities for advancing reinforcement learning (RL). This growth has motivated the development of passive RL, seeking to convert passive observations into actionable insights. This paper explores the prerequisites and mechanisms through which passive data can be utilized to improve online RL. We show that, in identifiable dynamics, where action impact can be distinguished from stochasticity, learning on passive data is statistically beneficial. Building upon the theoretical insights, we propose a novel algorithm named Multiscale State-Centric Planners (MSCP) that leverages two planners at distinct scales to offer guidance across varying levels of abstraction. The algorithm’s fast planner targets immediate objectives, while the slow planner focuses on achieving longer-term goals. Notably, the fast planner incorporates pessimistic regularization to address the distributional shift between offline and online data. MSCP effectively handles the practical challenges involving imperfect pretraining and limited dataset coverage. Our empirical evaluations across multiple benchmarks demonstrate that MSCP significantly outperforms existing approaches, underscoring its proficiency in addressing complex, long-horizon tasks through the strategic use of passive data.

Multi-core platforms are becoming increasingly prevalent in cyber-physical systems such as automobiles and robots. However, contention for shared resources makes it chal-lenging to guarantee timing predictability. Existing studies have primarily focused on characterizing the extent to which such interference can induce delays (usually from an adversarial perspective). Unfortunately, less is understood on the physical impacts of these timing delays in different cyber-physical plat-forms. In this paper, we fill this gap by providing an empirical examination of the end-to-end effects of performance interference on real-world applications. We analyze the root causes of harmful interference and summarize potential implementation pitfalls. To automate this process, we introduce TimeTrap, a tool that analyzes performance interference in autonomous systems through the lens of control outcome. To understand the extent to which timing interference may cause control deviations, TimeTrap has to strategically leverage different magnitudes of resource contention to trigger targeted deadline miss patterns. Through this exercise, we found that a naive approach that maximizes task latency via performance interference may fail to trigger worst-case outcomes (i.e. physical damages) due to built-in fail-safe mechanisms. As a result, delays have to be induced in a stealthy manner to avoid triggering fail-safes. To achieve this, TimeTrap first employs a system that actively injects fine-grained delays into the target software, adjusting the duration based on measured feedback. Second, TimeTrap leverages predictability in CPS execution patterns and resource usage to automatically tune its aggressor workloads, matching these patterns to achieve targeted interference and execution delays in a victim. We evaluate TimeTrap on two physical-world platforms and six platforms in a hardware-in-the-Ioop simulation environment, including robotic arms, UGVs, UAVs, self-driving cars, and humanoid robots. These studies demonstrate that an interference-based attack surface exists in different stages of the CPS pipeline, from perception to planning and control.

Elastic scheduling is a framework to reduce task utilizations (often by increasing periods) in response to system overload. This paper extends elastic scheduling to uniprocessor scheduling of implicit-deadline task sets for which periods must remain harmonic. We argue that for tasks with periods constrained to continuous intervals, the problem of selecting harmonic periods from those intervals is unlikely to have a polynomial time solution. However, we outline an approach that is pseudo-polynomial in the range of acceptable periods. We then show that the problem of elastic scheduling is NP-hard with harmonic constraints. Nonetheless, if a total order is imposed on task periods (a natural restriction in many applications with execution pipelines that synchronize input data sources), the problem can be reduced offline to a lookup table, enabling polynomial-time online adaptation if available CPU bandwidth changes. We implement the proposed algorithm in two real-world applications: the Fast Integrated Mobility Spectrometer (FIMS) and ORB-SLAM3. We demonstrate that elastic scheduling allows FIMS to adjust its execution to avoid missing deadlines on a SWaP-constrained computational platform, and that it improves ORB-SLAM3's localization results by as much as lO.4x when available CPU bandwidth changes dynamically during runtime.

Real-time and embedded systems are predominantly written in C, a language that is notoriously not memory safe. This has led to widespread memory-corruption vulnerabilities in real-time embedded cyber-physical systems (CPS). This is concerning, as such devices are becoming increasingly networked with the Internet of Things (IoT) and other communication technologies (e.g., 5G), rendering them vulnerable to remote attacks. Attackers have demonstrated how memory-corruption vulnerabilities can be used to hijack program control flow to implement arbitrary attacker-controlled logic. One class of defenses that has been developed to prevent such attacks is called control-flow integrity (CFI), which applies checks at control-flow transitions to ensure the target is valid. Unfortunately, attackers have shown how to divert control flow to seemingly valid targets in an invalid and malicious sequence. This paper presents InsectACIDE, the first holistic CFI for embedded and real-time systems that does not require binary instrumentation and that is context sensitive, i.e., it checks that the sequence of control-flow transitions taken is valid, not just individual transitions, thereby detecting such attacks. InsectACIDE is implemented on an embedded Cortex-M processor using the TrustZone trusted execution environment, and holistic context-sensitive CFI is enforced for both applications and the kernel. InsectACIDE uses hardware debugging features on the Cortex-M processor and therefore does not require any kernel or application binary modification. Experimental results show that InsectACIDE incurs significantly less runtime overhead compared to the state-of-the-art holistic CFI solution. Real-time schedulability analysis is presented, along with a schedulability evaluation, to demonstrate the tradeoff between stronger protection and real-time schedulability.

Safety- and mission-critical cyber-physical systems (CPSs) require temporal correctness to ensure safe physical behavior. This manifests as strict timing requirements, which cannot be missed at runtime. Counter-intuitively, this implies that real-time tasks can be delayed so long as they remain guaranteed to meet their deadlines. This paper explores how extra time in a schedule can be analytically recapitalized for the purpose of applying stronger security protection within individual tasks at compile time. This is achieved through the development of a partial context-sensitive pointer-integrity framework (ParCSPI). In this framework, more fine-grained policies can be enforced, with greater runtime overheads, where so doing does not violate real-time constraints. A whole-system optimization framework based upon a mixed-integer linear programming approach to fixed-priority response-time analysis is used to identify precisely which contexts can be checked within the available system-wide time while maximizing system-wide security. ParCSPI leverages Arm pointer authentication (PA) to encode context-based equivalence classes into the modifiers of the pointer signature and is implemented using a customized program analyzer and LLVM compiler passes. An evaluation of ParCSPI is presented that includes per-task and system-wide overhead and security tradeoffs, as well as a demonstration on a real-world CPS. Empirical results are presented showing that ParCSPI achieves up to 62% pointer-integrity protection with only 10% worst-case execution time (WCET) overhead, and can find optimal security trade-offs in complex real-time task sets as well as approximate them in reasonable time.

Explainability is increasingly recognized as an enabling technology for the broader adoption of machine learning (ML), particularly for safety-critical applications. This has given rise to explainable ML, which seeks to enhance the explainability of neural networks through the use of explanators. Yet, the pursuit for better explainability inadvertently leads to increased security and privacy risks. While there has been considerable research into the security risks of explainable ML, its potential privacy risks remain under-explored.To bridge this gap, we present a systematic study of privacy risks in explainable ML through the lens of membership inference. Building on the observation that, besides the accuracy of the model, robustness also exhibits observable differences among member samples and non-member samples, we develop a new membership inference attack. This attack extracts additional membership features from changes in model confidence under different levels of perturbations guided by the importance highlighted by the attribution maps in the explanators. Intuitively, perturbing important features generally results in a bigger loss in confidence for member samples. Using the member-non-member differences in both model performance and robustness, an attack model is trained to distinguish the membership. We evaluated our approach with seven popular explanators across various benchmark models and datasets. Our attack demonstrates there is non-trivial privacy leakage in current explainable ML methods. Furthermore, such leakage issue persists even if the attacker lacks the knowledge of training datasets or target model architectures. Lastly, we also found existing model and output-based defense mechanisms are not effective in mitigating this new attack.

Deep Convolution Neural Networks (CNNs) have become the cornerstone of image classification, but the emergence of adversarial image attacks brings serious security risks to CNN-based applications. As a local perturbation attack, the border attack can achieve high success rates by only modifying the pixels around the border of an image, which is a novel attack perspective. However, existing border attacks have shortcomings in stealthiness and are easily detected. In this article, we propose a novel stealthy border attack method based on deep feature alignment. Specifically, we propose a deep feature alignment algorithm based on style transfer to guarantee the stealthiness of adversarial borders. The algorithm takes the deep feature difference between the adversarial and the original borders as the stealthiness loss and thus ensures good stealthiness of the generated adversarial images. To ensure high attack success rates simultaneously, we apply cross entropy to design the targeted attack loss and use margin loss as well as Leaky ReLU to design the untargeted attack loss. Experiments show that the structural similarity between the generated adversarial images and the original images is 8.8% higher than the state-of-art border attack method, indicating that our proposed adversarial images have better stealthiness. At the same time, the success rate of our attack in the face of defense methods is much higher, which is about four times that of the state-of-art border attack under the adversarial training defense.

With the accelerating adaption of Cloud and Edge computing, cloud-based networked deployment emerges to enable providers to deliver services in a cost-effective and elastic manner. However, security concern remains one of the major obstacles to its wider adaption. Trusted Execution Environment (TEE) has been advocated to protect cloud services in an isolated execution environment. In this paper, we present a new genre of side-channel attack called interface-based side-channel attack and demonstrate its effectiveness on the TEE-assisted networked service system. The root cause of this attack is the input-dependent interface invocation (e.g., interface information and invocation patterns) that can be observed by untrusted software to reveal the control flows inside the enclave. Our evaluation demonstrates that the attack can effectively re-identify encrypted web pages processed in the SGX enclave with an accuracy of 87.6% and a recall of 76.6%, and can reduce the search domain of the 1024 bits RSA private keys to$1.69 \times 10^{-6}$of the original search domain. As countermeasures, we propose, implement and evaluate a set of static analysis tools to mitigate the newly discovered threats. The key idea is to use inter-procedural dataflow analysis to identify potential leakage via the interface, and then mitigate them during compilation using techniques including branch obfuscation, loop obfuscation, and constant size wrapper.

Graphics Processing Units (GPU) are increasingly deployed on Cyber-physical Systems (CPSs), frequently used to perform real-time safety-critical functions, such as object detection on autonomous vehicles. As a result, availability is important for GPU tasks in CPS platforms. However, existing Trusted Execution Environments (TEE) solutions with availability guarantees focus only on CPU computing.

The rapid development of deep neural networks and generative AI has catalyzed growth in realistic speech synthesis. While this technology has great potential to improve lives, it also leads to the emergence of ''DeepFake'' where synthesized speech can be misused to deceive humans and machines for nefarious purposes. In response to this evolving threat, there has been a significant amount of interest in mitigating this threat by DeepFake detection.

The tenth ACM Workshop on Moving Target Defense (MTD) is held on November 26, 2023, in conjunction with the ACM Conference on Computer and Communications Security (CCS). The main objective of the workshop is to discuss novel randomization, diversification, and dynamism techniques for computer systems and network, new metric and analysis frameworks to assess and quantify the effectiveness of MTD, and discuss challenges and opportunities that such defenses provide. We have constructed an exciting and diverse program of six refereed papers, and two invited keynote talks that will provide the participants with a vibrant and thought-provoking set of ideas and insights.

LiDAR-based perception is a central component of autonomous driving, playing a key role in tasks such as vehicle localization and obstacle detection. Since the safety of LiDAR-based perceptual pipelines is critical to safe autonomous driving, a number of past efforts have investigated its vulnerability under adversarial perturbations of raw point cloud inputs. However, most such efforts have focused on investigating the impact of such perturbations on predictions (integrity), and little has been done to understand the impact on latency (availability), a critical concern for real-time cyber-physical systems. We present the first systematic investigation of the availability of LiDAR detection pipelines, and SlowLiDAR, an adversarial perturbation attack that maximizes LiDAR detection runtime. The attack overcomes the technical challenges posed by the non-differentiable parts of the LiDAR detection pipelines by using differentiable proxies and uses a novel loss function that effectively captures the impact of adversarial perturbations on the execution time of the pipeline. Extensive experimental results show that SlowLiDAR can significantly increase the latency of the six most popular LiDAR detection pipelines while maintaining imperceptibility11Code is available at: https://github.com/WUSTL-CSPL/SlowLiDAR.

The field of text-to-image generation has made remarkable strides in creating high-fidelity and photorealistic images. As this technology gains popularity, there is a growing concern about its potential security risks. However, there has been limited exploration into the robustness of these models from an adversarial perspective. Existing research has primarily focused on untargeted settings, and lacks holistic consideration for reliability (attack success rate) and stealthiness (imperceptibility). In this paper, we propose RIATIG, a reliable and imperceptible adversarial attack against text-to-image models via inconspicuous examples. By formulating the example crafting as an optimization process and solving it using a genetic-based method, our proposed attack can generate imperceptible prompts for text-to-image generation models in a reliable way. Evaluation of six popular text-to-image generation models demonstrates the efficiency and stealthiness of our attack in both white-box and black-box settings. To allow the community to build on top of our findings, we've made the artifacts available11Code is available at: https://github.com/WUSTL-CSPL/RIATIG.

ARM Cortex-M is one of the most popular microcontroller architectures designed for embedded and Internet of Things (IoT) applications. To facilitate efficient execution, it has some unique hardware optimization. In particular, Cortex-M TrustZone has a fast state switch mechanism that allows direct control-flow transfer from the secure state program to the non-secure state userspace program. In this paper, we demonstrate how this fast state switch mechanism can be exploited for arbitrary code execution with escalated privilege in the non-secure state by introducing a new exploitation technique, namely return-to-non-secure (ret2ns). We experimentally confirmed the feasibility of four variants of ret2ns attacks on two Cortex-M hardware systems. To defend against ret2ns attacks, we design two address sanitizing mechanisms that have negligible performance overhead.

Tiny machine learning (TinyML) is an essential component of emerging smart microcontrollers (MCUs). However, the protection of the intellectual property (IP) of the model is an increasing concern due to the lack of desktop/server-grade resources on these power-constrained devices. In this paper, we propose STML, a system and algorithm co-design to Secure IP of TinyML on MCUs with ARM TrustZone. Our design jointly optimizes memory utilization and latency while ensuring the security and accuracy of emerging models. We implemented a prototype and benchmarked with 7 models, demonstrating STML reduces 40% of model protection runtime overhead on average.

Witnessing the blooming adoption of push notifications on mobile devices, this new message delivery paradigm has become pervasive in diverse applications. Accompanying with its broad adoption, the potential security risks and privacy exposure issues raise public concerns regarding its great social impacts. This paper conducts the first attempt to exploit the mobile notification ecosystem. By dissecting its structural elements and implementation process, a comprehensive vulnerability analysis is conducted towards the complete flow of mobile notification from platform enrollment to messaging. Meanwhile, for privacy exposure, we first examine the implementation of privacy policy compliance by proposing a three-level inspection approach to guide our analysis. Then, our top-down methods from documentation analysis, application network traffic study, to static analysis expose the illicit data collection behaviors in released applications. In addition, we uncover the potential privacy inference resulted from the notification monitoring. To support our analysis, we conduct empirical studies on 12 most popular notification platforms and perform static analysis over 30,000+ applications. We discover: 1) six platforms either provide ambiguous KEY naming rules or offer vulnerable messaging APIs; 2) privacy policy compliance implementations are either stagnated at the documentation stages (8 of 12 platforms) or never implemented in apps, resulting in billions of users suffering from privacy exposure; and 3) some apps can stealthily monitor notification messages delivering to other apps, potentially incurring user privacy inference risks. Our study raises the urgent demand for better regulations of mobile notification deployment.

Recent advances in large language models (LMs) have facilitated their ability to synthesize programming code. However, they have also raised concerns about intellectual property (IP) rights violations. Despite the significance of this issue, it has been relatively less explored. In this paper, we aim to bridge the gap by presenting CodeIPPrompt, a platform for automatic evaluation of the extent to which code language models may reproduce licensed programs. It comprises two key components: prompts constructed from a licensed code database to elicit LMs to generate IP-violating code, and a measurement tool to evaluate the extent of IP violation of code LMs. We conducted an extensive evaluation of existing open-source code LMs and commercial products and revealed the prevalence of IP violations in all these models. We further identified that the root cause is the substantial proportion of training corpus subject to restrictive licenses, resulting from both intentional inclusion and inconsistent license practice in the real world. To address this issue, we also explored potential mitigation strategies, including fine-tuning and dynamic token filtering. Our study provides a testbed for evaluating the IP violation issues of the existing code generation platforms and stresses the need for a better mitigation strategy.

When a blockchain application runs on data from the real world, it relies on an oracle mechanism that transports data from external sources to the blockchain. The blockchain oracle problem arises around the need to procure trustworthy data from external sources. Previous works have addressed data authenticity/integrity by building a secure channel between blockchain and external sources while employing a decentralized oracle network to avoid a single point of failure. However, the truthful data challenge, which emerges when legitimate external sources submit fraudulent or deceitful data, remains unsolved. In this paper, we introduce a new decentralized truth-discovering oracle architecture called DecenTruth to address the truthful data challenge using a data-centric approach. DecenTruth aims to elevate the "truthfulness" of external data input by enabling decentralized oracle nodes to discover and reach consensus on truthful values of common data objects from multi-sourced inputs in an off-chain manner. It harmonizes techniques in both the data plane and consensus plane—truth discovery (TD) and asynchronous BFT consensus—and enables nodes to finalize the same estimated truths on data objects with high accuracy, amid the harsh asynchronous network condition and presence of Byzantine sources and nodes. We implemented DecenTruth and evaluated its performance in a simulated oracle service scenario. The results demonstrate significantly higher Byzantine resilience and long-term data feed accuracy of DecenTruth, compared to existing median-based aggregation methods.

The Butterfly Attack, introduced in an RTSS 2019 paper, was billed as a new kind of timing attack against control loops in cyber-physical systems. We conduct a close inspection of the Butterfly Attack in order to identify the root vulnerability that it exploits, and show that an appropriate application of real-time scheduling theory provides an effective countermeasure. We propose improved defenses against this and similar attacks by drawing upon techniques from real-time scheduling theory, control theory, and systems implementation, that are both provably secure and are able to make efficient use of computing resources.

Time-sensitive applications, such as 5G and IoT, are imposing increasingly stringent security and reliability requirements on network time synchronization. Precision time protocol (PTP) is a de facto solution to achieve high precision time synchronization. It is widely adopted by many industries. Existing efforts in securing the PTP focus on the protection of communication channels, but little attention has been given to the threat of malicious insiders. In this paper, we first present the security vulnerabilities of PTP and discuss why the current defense mechanisms are unable to counter Byzantine insiders. We demonstrate how a malicious insider can spoof a time source to arbitrarily shift the system time of a victim node on an IoT testbed. We further demonstrate the harmful consequence of the attack on a real Turtlebot3 robotic platform as the robot fails to locate itself and follows a false trajectory. As a countermeasure, we propose multi-source PTP, in short, MS-PTP, a Byzantine-resilient network time synchronization mechanism that relies on time crowdsourcing. MS-PTP changes the current PTP's single source hierarchy to a multi-source client-server architecture, in which PTP clients take responses from multiple time servers and apply a novel secure aggregation scheme to eliminate the effect of malicious responses from unreliable sources. MS-PTP is able to counter f Byzantine failures when the total number of time sources n used by a client satisfies n>=3f+1. We provide rigorous proof for its non-parametric accuracy guarantee---achieving bounded error regardless of the Byzantine population. We implemented a prototype of MS-PTP on our IoT testbed and the results show its resilience against Byzantine insiders while maintaining high synchronization accuracy.

Federated meta-learning has emerged as a promising AI framework for today’s mobile computing scenes involving distributed clients. It enables collaborative model training using the data located at distributed mobile clients and accommodates clients that need fast model customization with limited new data. However, federated meta-learning solutions are susceptible to inference-based privacy attacks since the global model encoded with clients’ training data is open to all clients and the central server. Meanwhile, differential privacy (DP) has been widely used as a countermeasure against privacy inference attacks in federated learning. The adoption of DP in federated meta-learning is complicated by the model accuracy-privacy trade-off and the model hierarchy attributed to the meta-learning component. In this paper, we introduce DP-FedMeta, a new differentially private federated meta-learning architecture that addresses such data privacy challenges. DP-FedMeta features an adaptive gradient clipping method and a one-pass meta-training process to improve the model utility-privacy trade-off. At the core of DP-FedMeta are two DP mechanisms, namely DP-AGR and DP-AGRLR, to provide two notions of privacy protection for the hierarchical models. Extensive experiments in an emulated federated meta-learning scenario on well-known datasets (Omniglot, CIFAR-FS, and Mini-ImageNet) demonstrate that DP-FedMeta accomplishes better privacy protection while maintaining comparable model accuracy compared to the state-of-the-art solution that directly applies DP-based meta-learning to the federated setting.

As the COVID-19 pandemic fundamentally reshaped the remote life and working styles, Voice over IP (VoIP) telephony and video conferencing have become a primary method of connecting communities together. However, little has been done to understand the feasibility and limitations of delivering adversarial voice samples via such communication channels.

Unlawful video surveillance of unsuspecting individuals using spy cameras has become an increasing concern. To mitigate these threats, there are both commercial products and research prototypes designed to detect hidden spy cameras in household and office environments. However, existing work often relies heavily on user expertise and only applies to wireless cameras. To bridge this gap, we propose HeatDeCam, a thermal-imagery-based spy camera detector, capable of detecting hidden spy cameras with or without built-in wireless connectivity. To reduce the reliance on user expertise, HeatDeCam leverages a compact neural network deployed on a smartphone to recognize unique heat dissipation patterns of spy cameras. To evaluate the proposed system, we have collected and open-sourced a dataset of a total of 22506 thermal and visual images. These images consist of 11 spy cameras collected from 6 rooms across different environmental conditions. Using this dataset, we found HeatDeCam can achieve over 95% accuracy in detecting hidden cameras. We have also conducted a usability evaluation involving a total of 416 participants using both an online survey and an in-person usability test to validate HeatDeCam.

Microcontroller-based embedded systems have become ubiquitous with the emergence of IoT technology. Given its critical roles in many applications, its security is becoming increasingly important. Unfortunately, MCU devices are especially vulnerable. Code reuse attacks are particularly noteworthy since the memory address of firmware code is static. This work seeks to combat code reuse attacks, including ROP and more advanced JIT-ROP via continuous randomization. Previous proposals are geared towards full-fledged OSs with rich runtime environments, and therefore cannot be applied to MCUs. We propose the first solution for ARM-based MCUs. Our system, named HARM, comprises a secure runtime and a binary analysis tool with rewriting module. The secure runtime, protected inside the secure world, proactively triggers and performs non-bypassable randomization to the firmware running in a sandbox in the normal world. Our system does not rely on any firmware feature, and therefore is generally applicable to both bare-metal and RTOS-powered firmware. We have implemented a prototype on a development board. Our evaluation results indicate that HARM can effectively thaw code reuse attacks while keeping the performance and energy overhead low.

The wired ghost touch attacks are the emerging and severe threats against modern touchscreens. The attackers can make touchscreens falsely report nonexistent touches (i.e., ghost touches) by injecting common-mode noise (CMN) into the target devices via power cables. Existing attacks rely on reverse-engineering the touchscreens, then manually crafting the CMN waveforms to control the types and locations of ghost touches. Although successful, they are limited in practicality and attack capability due to the touchscreens' black-box nature and the immense search space of attack parameters. To overcome the above limitations, this paper presents PowerTouch, a framework that can automatically generate wired ghost touch attacks. We adopt a software-hardware co-design approach and propose a domain-specific genetic algorithm-based method that is tailored to account for the characteristics of the CMN waveform. Based on the security objectives, our framework automatically optimizes the CMN waveform towards injecting the desired type of ghost touches into regions specified by attackers. The effectiveness of PowerTouch is demonstrated by successfully launching attacks on touchscreen devices from two different brands given nine different objectives. Compared with the state-of-the-art attack, we seminally achieve controlling taps on an extra dimension and injecting swipes on both dimensions. We can place an average of 84.2% taps on the targeted side of the screen, with the location error in the other dimension no more than 1.53mm. An average of 94.5% of injected swipes with correct directions is also achieved. The quantitative comparison with the state-of-the-art method shows that a better attack performance can be achieved by PowerTouch.

Timing is an important property for robotic systems that continuously interact with our physical world. Variation in program execution time caused by limited computational resources or system resource contention can lead to significant impact on algorithmic result accuracy. Even though recent work has found Simultaneous Localization And Mapping (SLAM) to be timing-sensitive, little exists in understanding the interactions between the timing variations in SLAM systems and the corresponding degradation. In this paper we conduct a systematic analysis of nine state-of-the-art SLAM systems and dissect the root causes of their degradation. We discovered that timing-induced errors are generated either from delayed execution in certain critical tasks, or from desynchronization in sensor fusion. Based on the insights from our analysis, we propose a solution that combines selective fusion on data in the front end and temporal budget optimization on bundle adjust-ment in the backend to mitigate the impacts of unexpected timing variation adaptively. Experimental results show that our proposed method makes it possible to migrate expensive algorithms to low-cost platforms without laborious tuning, while making the SLAM system robust against the effects of abnormal timing.

Cybersecurity education is becoming increasingly important as demand for cybersecurity professionals increases. Hands-on skills are a critical component of cybersecurity education, and a variety of exercise types have been developed to teach these skills. In this work, we seek to apply the benefits of gamified learning to an introductory cybersecurity curriculum in the form of a set of Capture the Flag (CTF) challenges offered as hands-on exercises for an intro-level course. We created 20 jeopardy-style challenges of varying difficulty based on prior research on the use of gamification in education, and we configured the open-source CTFd platform to host our challenges. Student responses to post-challenge surveys suggest that the CTF component of the course was effective in improving perceived learning and student engagement.

As cyber-physical systems have become increasingly complex, rising computational demand has led to the ubiquitous use of multicore processors in embedded environments. Size, Weight, Power, and Cost (SWaP-C) constraints have pushed more processes onto shared platforms, including real-time tasks with deadline requirements. To prevent temporal interference among tasks running concurrently or in parallel in such systems, many operating systems provide priority-based scheduling and enforce processor reservations based on Worst-Case Execution Time (WCET) estimates. However, shared resources (both architectural components and data structures within the operating system) provide channels through which these constraints can be broken. Prior work has demonstrated that malicious execution by one or more processes can cause significant delays, leading to potential deadline misses in victim tasks. In this paper, we introduce PolyRhythm, a three-phase attack template that combines primitives across multiple architectural and kernel-based channels: (1) it uses an offline genetic algorithm to tune attack parameters based on the target hardware and OS platform; then (2) it performs an online search for regions of the attack parameter space where contention is most likely; and finally (3) it runs the attack primitives, using online reinforcement learning to adapt to dynamic execution patterns in the victim task. On a representative platform (Raspberry Pi 3B) Poly Rhythm outperforms prior work, achieving significantly more slowdown. As we show for several hardware/software platforms, Poly Rhythm also allows us to characterize the extent to which interference can occur; this helps to inform better estimates of execution times and overheads, towards preventing deadline misses in real-time systems.

The proliferation of real-time cyber-physical systems (CPS) is making profound changes to our daily life. Many real-time CPSs are security and safety-critical because of their continuous interactions with the physical world. While the general perception is that the security protection mechanism deployment is often absent in real-time embedded systems, there is no existing empirical study that measures the adoption of these mechanisms in the ecosystem. To bridge this gap, we conduct a measurement study for real-time embedded firmware from both a security perspective and a real-time perspective. To begin with, we collected more than 16 terabytes of embedded firmware and sampled 1,000 of them for the study. Then, we analyzed the adoption of security protection mechanisms and their potential impacts on the timeliness of real-time embedded systems. Besides, we measured the scheduling algorithms supported by real-time embedded systems since they are also security-critical.

Embedded devices are becoming increasingly pervasive in safety-critical systems of the emerging cyber-physical world. While trusted execution environments (TEEs), such as ARM TrustZone, have been widely deployed in mobile platforms, little attention has been given to deployment on real-time cyber-physical systems, which present a different set of challenges compared to mobile applications. For safety-critical cyber-physical systems, such as autonomous drones or automobiles, the current TEE deployment paradigm, which focuses only on confidentiality and integrity, is insufficient. Computation in these systems also needs to be completed in a timely manner (e.g., before the car hits a pedestrian), putting a much stronger emphasis on availability.To bridge this gap, we present RT-TEE, a real-time trusted execution environment. There are three key research challenges. First, RT-TEE bootstraps the ability to ensure availability using a minimal set of hardware primitives on commodity embedded platforms. Second, to balance real-time performance and scheduler complexity, we designed a policy-based event-driven hierarchical scheduler. Third, to mitigate the risks of having device drivers in the secure environment, we designed an I/O reference monitor that leverages software sandboxing and driver debloating to provide fine-grained access control on peripherals while minimizing the trusted computing base (TCB).We implemented prototypes on both ARMv8-A and ARMv8-M platforms. The system is tested on both synthetic tasks and real-life CPS applications. We evaluated rover and plane in simulation and quadcopter both in simulation and with a real drone.

Modern AI tools, such as generative adversarial networks, have transformed our ability to create and modify visual data with photorealistic results. However, one of the deleterious side-effects of these advances is the emergence of nefarious uses in manipulating information in visual data, such as through the use of deep fakes. We propose a novel architecture for preserving the provenance of semantic information in images to make them less susceptible to deep fake attacks. Our architecture includes semantic signing and verification steps. We apply this architecture to verifying two types of semantic information: individual identities (faces) and whether the photo was taken indoors or outdoors. Verification accounts for a collection of common image transformation, such as translation, scaling, cropping, and small rotations, and rejects adversarial transformations, such as adversarially perturbed or, in the case of face verification, swapped faces. Experiments demonstrate that in the case of provenance of faces in an image, our approach is robust to black-box adversarial transformations (which are rejected) as well as benign transformations (which are accepted), with few false negatives and false positives. Background verification, on the other hand, is susceptible to black-box adversarial examples, but be-comes significantly more robust after adversarial training.

Timing property plays a vital role in the Cyber-Physical System(CPS) due to its interaction with the physical world. The smooth operation of these robotic systems often relies on an accurate and timely perception and actuation of the physical world. In this poster, we demonstrated a unique new class of attack, Chronos, that exploits timing interference to cause system destabilization in cyber-physical systems. Using a compromised non-privileged non-critical task on the system, we launch timing interference attacks on both drone and autonomous vehicle platforms. Through both open-loop and close-loop testing on the end-to-end stack, we showed that the timing attack could lead to complete loss of control of the autonomous system, crashing them onto the surroundings when there is no software vulnerability. To further understand this novel attack vector, we perform preliminary investigations on the localization component of these two platforms, because they both make use of well-known simultaneous localization and mapping (SLAM) algorithms that depend on timing-sensitive multimodal data from different sensors. Building on the insights from the case study, we present our formulation of the timing attack surface and highlight future directions.

The growing complexity of our regulatory environment presents us with a hard problem: how can we determine if we are compliant with an ever-growing body of regulations? Computational legal auditing may help, as computational tools are exceptionally good at making sense of large amounts of data. In this research, we explore the possibility of creating a computational auditor that checks if mobile health (mHealth) apps satisfy federal security and privacy regulations. In doing so, we find that while it is challenging to convert open-ended, generally applicable, complicated laws into computational principles, the use of non-legal, authoritative, explanatory documents allows for computational operationalization while preserving the open-ended nature of the law. We test our auditor on 182 FDA/CE-approved mHealth apps. Our research suggests that the use of non-legal, authoritative, guidance documents may help with the creation of computational auditors, a promising tool to help us manage our ever-growing regulatory responsibilities.

Mobile devices are becoming an indispensable part of work for corporations and governments to store and process sensitive information. Thus, it is important for remote administrators to maintain control of these devices via Mobile Device Management (MDM) solutions. ARM TrustZone has been widely regarded as the de facto solution for protecting the security-sensitive software, such as MDM agents, from attacks of a compromised rich OS. However, little attention has been given to protecting the MDM control channel, a fundamental component for a remote administrator to invoke the TrustZone-based MDM agents and perform specific management operations. In this work, we design an ARM TrustZone-based network mechanism, called TZNIC, towards enabling resilient and secure access to TrustZone-based software, even in the presence of a malicious rich OS. TZNIC deploys two NIC drivers, one secure-world driver and one normal-world driver, multiplexing one physical NIC. We utilize the ARM TrustZone-based high privilege to protect the secure-world driver and further resolve several challenges on sharing one set of hardware peripherals between two isolated software environments. TZNIC does not require any changes or collaboration of the rich OS. We implement a prototype of TZNIC, and the evaluation results show that TZNIC can provide a reliable network channel to invoke the security software in the secure world, with minimal system overhead on the rich OS.

Detection of malicious behavior is a fundamental problem in security. One of the major challenges in using detection systems in practice is in dealing with an overwhelming number of alerts that are triggered by normal behavior (the so-called false positives), obscuring alerts resulting from actual malicious activities. We introduce a novel approach for computing a policy for prioritizing alerts using adversarial reinforcement learning. Our approach assumes that the attacker knows the full state of the detection system and the defender's alert prioritization policy, and will dynamically choose an optimal attack. The first step of our approach is to capture the interaction between the defender and attacker in a game theoretic model. To tackle the computational complexity of solving this game to obtain a dynamic stochastic alert prioritization policy, we propose an adversarial reinforcement learning framework. In this framework, we use neural reinforcement learning to compute best response policies for both the defender and the adversary to an arbitrary stochastic policy of the other. We then use these in a double-oracle framework to obtain an approximate equilibrium of the game, which in turn yields a robust stochastic policy for the defender. We use case studies in network intrusion and fraud detection to demonstrate that our approach is effective in creating robust alert prioritization policies.1

In the past decade, Trusted Execution Environment (TEE) provided by ARM TrustZone is becoming one of the primary techniques for enhancing the security of mobile devices. The isolation enforced by TrustZone can protect the trusted applications running in the TEE against malicious software in the untrusted rich execution environment (REE). However, TrustZone cannot completely prevent vulnerabilities in trusted applications residing in the TEE, which can then be used to attack other trusted applications or even the trusted OS. Previously, a number of memory corruption vulnerabilities have been reported on different TAs, which are written in memory-unsafe languages like C.

Automotive communication networks, represented by the CAN bus, are acclaimed for enabling real-time communication between vehicular ECUs but also criticized for their lack of effective security mechanisms. Various attacks have demonstrated that this security deficit renders a vehicle vulnerable to adversarial control that jeopardizes passenger safety. A recent standardization effort led by AUTOSAR has provided general guidelines for developing next-generation automotive communication technologies with built-in security mechanisms. A key security mechanism is message authentication between ECUs for countering message spoofing and replay attack. While many message authentication schemes have been proposed by previous work, the important issue of session key establishment with AUTOSAR compliance was not well addressed. In this paper, we fill this gap by proposing an AUTOSAR-compliant key management architecture that takes into account practical requirements imposed by the automotive environment. Based on this architecture, we describe a baseline session key distribution protocol called SKDC that realizes all designed security functionalities, and propose a novel secret-sharing-based protocol called SSKT that yields improved communication efficiency. Both SKDC and SSKT are customized for CAN/CAN-FD bus deployment. We implemented the two protocols on commercial microcontroller boards and evaluated their performance with hardware experiment and extrapolation analysis. The result shows while both protocols are performant, SSKT achieves superior computation and communication efficiency at scale.

Big data analytics is having a profound impact on many sectors of the economy by transforming raw data into actionable intelligence. However, increased use of sensitive business and private personal data with no or limited privacy safeguards has raised great concerns among individuals and government regulators. To address the growing tension between the need for data utility and the demand for data privacy, trusted execution environment (TEE) is being used in academic research as well as industrial application as a powerful primitive to enable confidential computation on the private data with only the result disclosed but not the original private data. While much of the current research has been focusing on protecting the TEE against attacks (e.g. side-channel information leakage), the security and privacy of the applications executing inside a TEE enclave has received little attention. The general attitude is that the application is running inside a trusted computing base (TCB), and therefore can be trusted. This assumption may not be valid when it comes to unverified third-party applications. In this paper, we present PrivacyScope, a static code analyzer designed to detect leakage of private data by an application code running in a TEE. PrivacyScope accomplishes this by analyzing the application code and identifying violations of a property called nonreversibility. We introduce nonreversibility since the classical noninterference property falls short of detecting private data leakage in certain scenarios, e.g., in machine learning (ML) programs where the program output is always related to (private) input data. Given its strict reliance on observable state, the noninterference falls short of detecting private data leakage in these situations. By design, PrivacyScope detects both explicit and implicit information leakage. The nonreversibility property is formally defined based on the noninterference property. Additionally, we describe the algorithms for PrivacyScope as extensions to the runtime semantics of a general language. To evaluate the efficacy of our approach and proof-of-feasibility prototype, we apply PrivacyScope to detect data leakage in select open-source ML code modules including linear regression, k-means clustering and collaborative filtering. Also, PrivacyScope can detect intentional data leakage code injected by a programmer. We responsibly disclosed all the discovered vulnerabilities leading to disclosure of private data in the open-source ML program we analyzed.

Blockchain, the technology behind the popular Bitcoin, is considered a "security by design" system as it is meant to create security among a group of distrustful parties yet without a central trusted authority. The security of blockchain relies on the premise of honest-majority, namely, the blockchain system is assumed to be secure as long as the majority of consensus voting power is honest. And in the case of proof-of-work (PoW) blockchain, adversaries cannot control more than 50% of the network's gross computing power. However, this 50% threshold is based on the analysis of computing power only, with implicit and idealistic assumptions on the network and node behavior. Recent researches have alluded that factors such as network connectivity, presence of blockchain forks, and mining strategy could undermine the consensus security assured by the honest-majority, but neither concrete analysis nor quantitative evaluation is provided. In this paper we fill the gap by proposing an analytical model to assess the impact of network connectivity on the consensus security of PoW blockchain under different adversary models. We apply our analytical model to two adversarial scenarios: 1) honest-but-potentially-colluding, 2) selfish mining. For each scenario, we quantify the communication capability of nodes involved in a fork race and estimate the adversary's mining revenue and its impact on security properties of the consensus protocol. Simulation results validated our analysis. Our modeling and analysis provide a paradigm for assessing the security impact of various factors in a distributed consensus system.

In the era of ubiquitous intelligence, the Internet of Things (IoT) holds the promise as a breakthrough technology to enable diverse applications that benefit societal problems. Yet interconnecting myriad heterogeneous IoT devices across various application domains remain a security challenge. Decentralized technology has recently emerged as a powerful primitive in building distributed applications to facilitate secure transactions between mutually distrustful parties in a trustworthy manner. Unfortunately, these decentralized protocols demand computing resources and power far beyond the reach of resource-constrained IoT devices, preventing the full adoption of distributed consensus platform in the IoT setting. In this article, we address the key bottleneck to enable blockchain in resource-constrained IoT devices. We propose a lightweight implementation of proof-of-work (PoW) mining with reconfigurable hardware primitives. By replacing the hash and cryptographic functions in classic blockchain protocol with secure and efficient hardware implementations, our proposed solution can significantly reduce hardware resources and power overheads of PoW mining, while improving the transaction speed of large-scale IoT systems. Finally, we demonstrate the algorithm by proposing an antispoofing solution for GPS navigation among lightweight IoT devices. As a replacement for position computation, a mining process generates the expected coordinates with the correct initial value and function configuration.

On ARM processors with TrustZone security extension, asynchronous introspection mechanisms have been developed in the secure world to detect security policy violations in the normal world. These mechanisms provide security protection via passively checking the normal world snapshot. However, since previous secure world checking solutions require to suspend the entire rich OS, asynchronous introspection has not been widely adopted in the real world. Given a multi-core ARM system that can execute the two worlds simultaneously on different cores, secure world introspection can check the rich OS without suspension. However, we identify a new normal-world evasion attack that can defeat the asynchronous introspection by removing the attacking traces in parallel from one core when the security checking is performing on another core. We perform a systematic study on this attack and present its efficiency against existing asynchronous introspection mechanisms. As the countermeasure, we propose a secure and trustworthy asynchronous introspection mechanism called SATIN, which can efficiently detect the evasion attacks by increasing the attackers' evasion time cost and decreasing the defender's execution time under a safe limit. We implement a prototype on an ARM development board and the experimental results show that SATIN can effectively prevent evasion attacks on multi-core systems with a minor system overhead.

As an important application of the Internet of Things, many remote monitoring systems adopt a device-to-cloud network paradigm. In a remote patient monitoring case, various resource-constrained devices are used to measure the health conditions of a target patient in a distant non-clinical environment and the collected data are sent to the cloud backend of an authorized health care service for processing and decision making. As the measurements involve private patient information, access control and trustworthy processing of the confidential data become very important. Software-based solutions that adopt advanced cryptographic tools, such as attribute-based encryption and fully homomorphic encryption, can address the problem, but they also impose substantial computation overhead on both client and server sides. In this paper, we deviate from the conventional software-based solutions and propose a secure and efficient remote monitoring framework, called SRM, using the latest hardware-based trustworthy computing technology, such as Intel SGX. In addition, we present a robust and lightweight “heartbeat” protocol to handle notoriously difficult key revocation problem. We implemented a prototype of the framework for SRM and show that SRM can protect user data privacy against unauthorized parties, with minimum performance cost compared to existing software-based solutions.

Everyday billions of multimodal posts containing both images and text are shared in social media sites such as Snapchat, Twitter or Instagram.This combination of image and text in a single message allows for more creative and expressive forms of communication, and has become increasingly common in such sites.This new paradigm brings new challenges for natural language understanding, as the textual component tends to be shorter, more informal, and often is only understood if combined with the visual context.In this paper, we explore the task of name tagging in multimodal social media posts.We start by creating two new multimodal datasets: one based on Twitter posts 1 and the other based on Snapchat captions (exclusively submitted to public and crowdsourced stories).We then propose a novel model based on Visual Attention that not only provides deeper visual understanding on the decisions of the model, but also significantly outperforms other state-of-theart baseline methods for this task.2

With the emergence of Internet of Things, mobile devices are generating more network traffic than ever. TrustZone is a hardware-enabled trusted execution environment for ARM processors. While TrustZone is effective in providing the much-needed memory isolation, we observe that it is possible to derive secret information from secure world using the cache contention, due to its high-performance cache sharing design. In this work, we propose TruSense to study the timing-based cache side-channel information leakage of TrustZone. TruSense can be launched from not only the normal world operating system but also a non-privileged user application. Without access to virtual-to-physical address mapping in user applications, we devise a novel method that uses the expected channel statistics to allocate memory for cache probing. We also show how an attacker might use the less accurate performance event interface as a timer. Using the T-table based AES implementation in OpenSSL 1.0.1f as an example, we demonstrate how a normal world attacker can steal fine-grained secret in the secure world. We also discuss possible mitigations for the information leakage.

With increasingly complex cyber attacks occurring every day, memory-based forensic techniques are becoming instrumental in digital investigations. Forensic examiners can unravel what happened on a system by acquiring and inspecting in-memory data. However, the foundation of this analysis can be invalidated if the memory acquisition has been altered. In this paper, we study the feasibility of malicious software misusing architectural features to sabotage memory forensics. The misuse of two architectural features, namely, physical address layout and secure containers, is presented. The first architectural feature explored in this paper is the physical address layout. It is used by the northbridge to route memory access to either physical memory or I/O devices on x86 platforms. Observing this design choice, we propose Hidden in I/O Space (HIveS), which manipulates CPU registers to alter the physical address layout to conceal memory. The system uses a novel I/O shadowing technique to lock a memory region named HIveS memory into I/O address space to prevent access. Two novel techniques, blackbox write and TLB camouflage, are developed to further protect the unlocked HIveS memory against memory forensics while allowing access for attackers. The second architectural feature explored in this paper is hardware-aided secure execution technology. More specifically, hardware-enforced memory encryption in Intel secure guard extension is used in malicious enclave software (Malclaveware) to prevent introspection and memory forensics. A prototype of HIveS is built and tested against a set of memory acquisition tools for both Windows and Linux running on the x86 platform. Malclaveware is also prototyped in Windows to demonstrate the risk. More importantly, we proposed countermeasures and mitigations for the newly discovered attacks. Through these discussions, we aim to raise the awareness of the potential risks of misusing hardware architectural features.

As computing system continues to play an increasing role in daily life, user authentication is now an important component. One of the most widely accepted methods for user authentication is through proof of knowledge of a piece of secret information, such as password. However, entering this non-mutable secret for authentication in public space often allows attackers to steal the secret by shoulder surfing or video recording. We observe that it is possible to block attacker's access to user input using augmented reality (AR) display, which is only available to the user. Based on this intuition, we present AugAuth, an authentication scheme in AR using commercial off-the-shelf(COTS) gesture control sensors as an input device. AugAuth can resist against shoulder surfing by presenting user input interface that is only visible to the user and is unique every time. To enable user input with finger movement using the gesture control armband, Myo, we have solved several challenges in electromyogram signal processing, such as annotating the start of signal and finger classification. The experiment results for our input system of a group of volunteers show that our finger classification function has high accuracy and AugAuth is practical for use in real life authentication scenarios.

As the cost of human full genome sequencing continues to fall, we will soon witness a prodigious amount of human genomic data in the public cloud. To protect the confidentiality of the genetic information of individuals, the data has to be encrypted at rest. On the other hand, encryption severely hinders the use of this valuable information, such as Genome-wide Range Query (GRQ), in medical/genomic research. While the problem of secure range query on outsourced encrypted data has been extensively studied, the current schemes are far from practical deployment in terms of efficiency and scalability due to the data volume in human genome sequencing. In this paper, we investigate the problem of secure GRQ over human raw aligned genomic data in a third-party outsourcing model. Our solution contains a novel secure range query scheme based on multi-keyword symmetric searchable encryption (MSSE). The proposed scheme incurs minimal ciphertext expansion and computation overhead. We also present a hierarchical GRQ-oriented secure index structure tailored for efficient and large-scale genomic data lookup in the cloud while preserving the query privacy. Our experiment on real human genomic data shows that a secure GRQ request with range size 100,000 over more than 300 million encrypted short reads takes less than 3 minutes, which is orders of magnitude faster than existing solutions.

With the increasing popularity of augmented reality (AR) services, providing seamless human-computer interactions in the AR setting has received notable attention in the industry. Gesture control devices have recently emerged to be the next great gadgets for AR due to their unique ability to enable computer interaction with day-to-day gestures. While these AR devices are bringing revolutions to our interaction with the cyber world, it is also important to consider potential privacy leakages from these always-on wearable devices. Specifically, the coarse access control on current AR systems could lead to possible abuse of sensor data. Although the always-on gesture sensors are frequently quoted as a privacy concern, there has not been any study on information leakage of these devices. In this article, we present our study on side-channel information leakage of the most popular gesture control device, Myo. Using signals recorded from the electromyography (EMG) sensor and accelerometers on Myo, we can recover sensitive information such as passwords typed on a keyboard and PIN sequence entered through a touchscreen. EMG signal records subtle electric currents of muscle contractions. We design novel algorithms based on dynamic cumulative sum and wavelet transform to determine the exact time of finger movements. Furthermore, we adopt the Hudgins feature set in a support vector machine to classify recorded signal segments into individual fingers or numbers. We also apply coordinate transformation techniques to recover fine-grained spatial information with low-fidelity outputs from the sensor in keystroke recovery. We evaluated the information leakage using data collected from a group of volunteers. Our results show that there is severe privacy leakage from these commodity wearable sensors. Our system recovers complex passwords constructed with lowercase letters, uppercase letters, numbers, and symbols with a mean success rate of 91%.

With the growing importance of networked embedded devices in the upcoming Internet of Things, new attacks targeting embedded OSes are emerging. ARM processors, which power over 60% of embedded devices, introduce a hardware security extension called TrustZone to protect secure applications in an isolated secure world that cannot be manipulated by a compromised OS in the normal world. LeveragingTrustZone technology, a number of memory integrity checking schemes have been proposed in the secure world to introspect malicious memory modification of the normal world. In this paper, we first discover and verify an ARM TrustZone cache incoherence behavior, which results in the cache contents of the two worlds, secure and non-secure, potentially being different even when they are mapped to the same physical address. Furthermore, code in one TrustZone world cannot access the cache content in the other world. Based on this observation, we develop a new rootkit called CacheKit that hides in the cache of the normal world and is able to evade memory introspection from the secure world. We implement a CacheKit prototype on Cortex-A8 processors after solving a number of challenges. First, we employ the Cache-as-RAM technique to ensure that the malicious code is only loaded into the CPU cache and not RAM. Thus, the secure world cannot detect the existence of the malicious code by examining the RAM. Second, we use the ARM processor's hardware support on cache settings to keep the malicious code persistent in the cache. Third, to evade introspection that flushes cache content back into RAM, we utilize physical addresses from the I/O address range that is not backed by any real I/O devices or RAM. The experimental results show that CacheKit can successfully evade memory introspection from the secure world and has small performance impacts on the rich OS. We discuss potential countermeasures to detect this type of rootkit attack.

Recognizing the pressing demands to secure embedded applications, ARM TrustZone has been adopted in both academic research and commercial products to protect sensitive code and data in a privileged, isolated execution environment. However, the design of TrustZone cannot prevent physical memory disclosure attacks such as cold boot attack from gaining unrestricted read access to the sensitive contents in the dynamic random access memory (DRAM). A number of system-on-chip (SoC) bound execution solutions have been proposed to thaw the cold boot attack by storing sensitive data only in CPU registers, CPU cache or internal RAM. However, when the operating system, which is responsible for creating and maintaining the SoC-bound execution environment, is compromised, all the sensitive data is leaked. In this paper, we present the design and development of a cache-assisted secure execution framework, called CaSE, on ARM processors to defend against sophisticated attackers who can launch multi-vector attacks including software attacks and hardware memory disclosure attacks. CaSE utilizes TrustZone and Cache-as-RAM technique to create a cache-based isolated execution environment, which can protect both code and data of security-sensitive applications against the compromised OS and the cold boot attack. To protect the sensitive code and data against cold boot attack, applications are encrypted in memory and decrypted only within the processor for execution. The memory separation and the cache separation provided by TrustZone are used to protect the cached applications against compromised OS. We implement a prototype of CaSE on the i.MX53 running ARM Cortex-A8 processor. The experimental results show that CaSE incurs small impacts on system performance when executing cryptographic algorithms including AES, RSA, and SHA1.

With the growing complexity of computing systems, memory based forensic techniques are becoming instrumental in digital investigations. Digital forensic examiners can unravel what happened on a system by acquiring and inspecting in-memory data. Meanwhile, attackers have developed numerous anti-forensic mechanisms to defeat existing memory forensic techniques by manipulation of system software such as OS kernel. To counter anti-forensic techniques, some recent researches suggest that memory acquisition process can be trusted if the acquisition module has not been tampered with and all the operations are performed without relying on any untrusted software including the operating system.

With recent advances in communication and data storage technology, an explosive amount of information is being collected and stored in the Internet. Even though such vast amount of information presents great opportunities for knowledge discovery, organizations might not want to share their data due to legal or competitive reasons. This posts the challenge of mining knowledge while preserving privacy. Current efficient privacy-preserving data mining algorithms are based on an assumption that it is acceptable to release all the intermediate results during the data mining operations. However, it has been shown that such intermediate results can still leak private information. In this work, we use differential privacy to quantitatively limit such information leak. Differential privacy is a newly emerged privacy definition that is capable of providing strong measurable privacy guarantees. We propose Secure group Differential private Query (SDQ), a new algorithm that combines techniques from differential privacy and secure multiparty computation. Using decision tree induction as a case study, we show that SDQ can achieve stronger privacy than current efficient secure multiparty computation approach, and better accuracy than current differential privacy approach while maintaining efficiency.