Training and fine-tuning large-scale language models largely benefit from collaborative learning, but the approach has been proven vulnerable to gradient inversion attacks (GIAs), which allow adversaries to reconstruct private training data from shared gradients. Existing defenses mainly employ gradient perturbation techniques, e.g., noise injection or gradient pruning, to disrupt GIAs' direct mapping from gradient space to token space. However, these methods often fall short due to the retention of semantics similarity across gradient, embedding, and token spaces. Attackers can map proximate gradients into similar embeddings, and subsequently correspond them to tokens with similar semantics.

Poisoning attacks have been shown to pose significant threats to federated learning (FL), including both untargeted and targeted attacks. To mitigate such threats, the majority of existing defenses are implemented on the server side during the aggregation process. However, as data remains inherently local in FL, these defenses either rely on unreliable statistical or structural properties of local updates, or make strong assumptions that rely on dataset information. As a result, the unique advantage of clients having access to trusted local data and training dynamics has been largely overlooked. In this work, we propose ClieND1, a novel client-side detection framework that shifts the detection of poisoning attacks from the server to the client. Specifically, ClieND enables each client to maintain the neuron importance scores of the aggregated global model in each round by leveraging its local dataset. Through tracking the inter-round changes in these scores, each client detects abnormal behavior by identifying significant discrepancy spikes, which serve as indicators of potential poisoning attacks. To evaluate the effectiveness of ClieND, we compare it against three state-of-the-art baselines under both targeted and untargeted poisoning attacks, and also assess its ability to detect attacks at an early stage. Experimental results show that ClieND achieves a false positive rate (FPR) as low as 0.01 and a true positive rate (TPR) of up to 0.99, which significantly outperforms server-side defenses, demonstrating its high detection accuracy. Additionally, the existence of poisoning attacks, even those launched by adaptive settings, can be detected in an early stage within 5 communication rounds.

Split Learning (SL) offers a framework for collaborative model training that respects data privacy by allowing participants to share the same dataset while maintaining distinct feature sets. However, SL is susceptible to backdoor attacks, in which malicious clients subtly alter their embeddings to insert hidden triggers that compromise the final trained model. To address this vulnerability, we introduce SecureSplit, a defense mechanism tailored to SL. SecureSplit applies a dimensionality transformation strategy to accentuate subtle differences between benign and poisoned embeddings, facilitating their separation. With this enhanced distinction, we develop an adaptive filtering approach that uses a majority-based voting scheme to remove contaminated embeddings while preserving clean ones. Rigorous experiments across four datasets (CIFAR-10, MNIST, CINIC-10, and ImageNette), five backdoor attack scenarios, and seven alternative defenses confirm the effectiveness of SecureSplit under various challenging conditions.

Abstract Training deep learning models requires substantial financial and human resources, so once deployed in untrusted environments, these models immediately attract the attention of attackers who seek to steal and misuse them. Traditional model protection methods are ineffective in addressing model accuracy, performance, and proactive defense. To this end, we present an active defensive approach SwitchNet by obfuscating model structure and proposing a switch-controlled mechanism to manage model inference. Specifically, SwitchNet learns the weight distribution of the original model and then constructs confusion layers that are strategically inserted into the original model for structure obfuscation. Each of the model layers is equipped with a switch , which is controlled by a switching policy network. We train this policy network with an adaptive pattern as a “secret key” that can accurately control the switch states, and thereby the model inference process. We conduct a comprehensive theoretical analysis of the perturbation boundary and certify that SwitchNet maintains high robustness under $$\ell _\infty$$ ℓ ∞ perturbations, with certified accuracy exceeding 80% at $$\epsilon = 0.0048$$ ϵ = 0.0048 (CROWN). In addition, we perform extensive experiments on both classical convolutional networks and Vision Transformers. The results show that SwitchNet effectively preserves model accuracy for legitimate users (with only a 0.35% drop), while reducing the accuracy for unauthorized users to near-random guessing. Compared to the state-of-the-art, our approach reduces inference and construction overhead by 20.89% and 12.08%, respectively. Furthermore, SwitchNet proves to be stealthy and resilient against various attacks aimed at detecting or compromising the protection mechanism.

Mobile edge computing (MEC) is a promising paradigm that provides abundant computation and storage resources at the edge close to mobile devices (MDs). In MEC networks, MDs offload compute-heavy tasks to nearby edge servers (ESs) for delay-sensitive processing, where relevant services are stored to support task execution. However, the limited computation and storage capacities of ESs make joint optimization of service caching and computation offloading challenging due to coupled decisions, a large solution space, and dynamic environments. In this paper, we investigate the joint optimization of service caching and computation offloading in MEC networks, aiming to maximize the cache hit ratio and minimize the average service latency. To tackle this problem, the original formulation is decomposed into two hierarchical subproblems, namely high-level service caching and low-level computation offloading. We propose a novel hierarchical deep reinforcement learning (DRL) algorithm with active inference, termed HADRL. At the high-level, we adopt a deep deterministic policy gradient (DDPG) based DRL approach to maximize the cache hit ratio. At the low-level, we employ an active inference based DRL approach to minimize the average service latency. Unlike conventional DRL, the active inference based DRL approach selects policies by minimizing expected free energy instead of relying only on explicit rewards, making it well suited for highly dynamic low-level computation offloading. According to the simulation outcomes, the HADRL scheme surpasses the benchmark algorithms with respect to cache hit ratio as well as average service latency.

Modern deep face recognition systems are crucial for identity verification and authentication in high-stakes sectors such as security, finance, and law enforcement. However, their reliability is increasingly threatened by the rapid advancement of deepfake technology. Despite considerable attention, comprehensive evaluation of how emerging deepfake generation methods affect reliability of face recognition systems remains lacking. In this paper, we introduce a principled taxonomy and a benchmarking framework that structure deepfake attacks by intent and generative mechanism, enabling consistent and comprehensive assessment. Numerous results show that diverse deepfake attacks commonly exceed 70% success and, in some regimes, surpass 90%. Built on this foundation, we investigate the underlying mechanisms across different deepfake techniques, providing not only broad empirical evaluation but also valuable insight into what drives attack success. Our in-depth analysis reveals that success is not governed by visual quality, but rather by the degree of identity controllability.

Sharing personal photos on social media exposes users to unauthorized identity recognition and unconsented model training, raising severe privacy and copyright concerns. Existing methods typically focus on either privacy protection, which misleads recognition models to prevent unauthorized automated recognition, or tracing forensics, which embeds traceable patterns for ownership verification. However, they fail to achieve both simultaneously. The core challenge is to jointly achieve privacy protection and tracing forensics within a single perturbation, since the two objectives rely on different feature behaviors and naive combinations are ineffective in practice. In this work, we propose ATP (Adversarial Tracing Perturbation), a novel perturbation generation method that activates robustness-aware feature channels to balance privacy and traceability. ATP leverages non-robust channel activation to mislead recognition models for privacy protection, while robust channel activation embeds traceable patterns for reliable tracing forensics. Extensive experiments on image classification and face recognition show that ATP achieves strong dual protection, improving overall dual-protection performance by bm 3.54× over the baselines while remaining effective under adaptive attacks, thereby demonstrating strong robustness and practical applicability. © 2026 IEEE.

Enabling search over encrypted cloud data is essential for privacy-preserving data outsourcing. While searchable encryption has evolved to support individual requirements like fuzzy matching (tolerance to typos and variants in query keywords), dynamic updates, and result verification, designing a service that supports Dynamic Verifiable Fuzzy Search (DVFS) over encrypted cloud data remains a fundamental challenge due to inherent conflicts between underlying technologies. Existing approaches struggle with simultaneously achieving efficiency, functionality, and security, often forcing impractical trade-offs. This paper presentsVeriFuzzy, a novel DVFS service framework that cohesively integrates three innovations: anEnhanced Virtual Binary Tree (EVBTree)that decouples fuzzy semantics from index logic to support$O(\log n)$search/updates; ablockchain-reconstructed verificationmechanism that ensures result integrity with logarithmic complexity; and adual-repository state managementscheme that achieves IND-CKA2 security by neutralizing branch leakage. Extensive evaluation on 3,500+ documents shows VeriFuzzy achieves 41% faster search,$5\times$more efficient verification, and constant-time index updates compared to state-of-the-art alternatives. Our code and dataset are now open source, hoping to inspire future DVFS research.

The recent advancement of the automatic speech recognition (ASR) contributes to the voice user interface (VUI), which is broadly embedded into mobile apps. The VUI implemented on modern mobile operating systems like Android naturally involves multiple threads, and brings new race issues and challenges in defining and identifying them. Specifically, when the GUI and VUI (GV) actions both access to the same resource simultanously, the data race named GV-race may occur. GV-race can lead to wrong behavior and even crashes. However, to the best of our knowledge, this problem has not been adequately studied. In this paper, we present the first study of GV-race on Android apps. However, the involvement of the VUI complicates the concurrency model, affects the temporal relationship and brings state space explosion in global analysis. To tackle these challenges, we firstly defineprimitivesand theirhappen-beforerules to abstract GV interaction patterns. Using these primitives, we are able to characterize and formally define GV-race. We then developRoma(GV-race detectoronmobileapps) to detect both app-level and system-level GV-race automatically. Through static program analysis, Roma extracts GV related call graphs for each pair of conflicting GV actions to reduce the state space, and generates a universal GV interaction graph using our pre-defined primitives. It encodes happen-before constraints to formally specify thefreeness of GV-race, so that the detection of GV-race can be reduced to constraint solving with SMT solvers. We apply Roma to analyze 266 apps. Roma finds 52 apps with app-level GV-race and 56 apps with system-level GV-race. We confirm that 101 apps are true positives.

With large AI systems and models (LAMs) playing an ever-growing role across diverse applications, their impact on the privacy and cybersecurity of critical infrastructure has become a pressing concern. The LAMPS workshop is dedicated to tackling these emerging challenges, promoting dialogue on cutting-edge developments and ethical issues in safeguarding LAMs within critical infrastructure contexts. Bringing together leading experts from around the world, this workshop will delve into the complex privacy and cybersecurity risks posed by LAMs in critical sectors. Attendees will explore innovative solutions, exchange best practices, and contribute to shaping the future research agenda, emphasizing the crucial balance between advancing AI technologies and securing critical digital and physical infrastructures.

As healthcare systems evolve and healthcare data grows, the need for cross-domain collaboration treatment has become more complex, necessitating fine-grained access control to enhance privacy and security. Blockchain provides a distributed trusted platform without third parties, but the current blockchain-based access control systems lack efficiency and sufficient privacy protection in cross-domain collaboration. To address these challenges, we propose SWIFTGUARD, an efficient and fine-grained access control system based on a master-slave chain to strengthen the security and privacy of cross-domain healthcare collaboration. SWIFTGUARD incorporates a zero-knowledge proof protocol for cross-domain authentication with-out exposing sensitive data and leverages quantitative attribute weights for efficient access control. Through game-based security proof, we demonstrate the zero knowledge and soundness of the system. Extensive experiments evaluate that SWIFTGUARD reduces the time complexity of access authorization from O($n$) to O(log$n$), with improved throughput and stable performance in cross-domain collaboration. Our comprehensive evaluation confirms that SWIFTGUARD provides a secure and efficient access control system for cross-domain healthcare collaboration.

Out-of-distribution (OOD) detection remains challenging in text-rich networks, where textual features intertwine with topological structures.Existing methods primarily address label shifts or rudimentary domain-based splits, overlooking the intricate textual-structural diversity.For example, in social networks, where users represent nodes with textual features (name, bio) while edges indicate friendship status, OOD may stem from the distinct language patterns between bot and normal users.To address this gap, we introduce the Text-TopoOOD framework for evaluating detection across diverse OOD scenarios: (1) attributelevel shifts via text augmentations and embedding perturbations; (2) structural shifts through edge rewiring and semantic connections; (3) thematically-guided label shifts; and (4) domain-based divisions.Furthermore, we propose TNT-OOD to model the complex interplay between Text aNd Topology using: 1) a novel cross-attention module to fuse local structure into node-level text representations, and 2) a HyperNetwork to generate node-specific transformation parameters.This aligns topological and semantic features of ID nodes, enhancing ID/OOD distinction across structural and textual shifts.Experiments on 11 datasets across four OOD scenarios demonstrate the nuanced challenge of TextTopoOOD for evaluating OOD detection in text-rich networks.

Despite graph neural networks' (GNNs) great success in modelling graph-structured data, out-of-distribution (OOD) test instances still pose a great challenge for current GNNs. One of the most effective techniques to detect OOD nodes is to expose the detector model with an additional OOD node-set, yet the extra OOD instances are often difficult to obtain in practice. Recent methods for image data address this problem using OOD data synthesis, typically relying on pre-trained generative models like Stable Diffusion. However, these approaches require vast amounts of additional data, as well as one-for-all pre-trained generative models, which are not available for graph data. Therefore, we propose the GOLD framework for graph OOD detection, an implicit adversarial learning pipeline with synthetic OOD exposure without pre-trained models. The implicit adversarial training process employs a novel alternating optimisation framework by training: (1) a latent generative model to regularly imitate the in-distribution (ID) embeddings from an evolving GNN, and (2) a GNN encoder and an OOD detector to accurately classify ID data while increasing the energy divergence between the ID embeddings and the generative model's synthetic embeddings. This novel approach implicitly transforms the synthetic embeddings into pseudo-OOD instances relative to the ID data, effectively simulating exposure to OOD scenarios without auxiliary data. Extensive OOD detection experiments are conducted on five benchmark graph datasets, verifying the superior performance of GOLD without using real OOD data compared with the state-of-the-art OOD exposure and non-exposure baselines.

With the development of the Android system and increasing screen size, the use of multi-window mode has become prevalent among users. However, the security and privacy implications associated with this mode have not been thoroughly investigated. This paper uncovers severe and unique security vulnerabilities in Android's multi-window mode, revealing several high-risk side-channels that facilitate diverse cross-window attacks, leading to significant breaches of user privacy. In detail, our research introduces WinSpy, a framework leveraging a newly discovered resource contention side-channel in multi-window mode to fingerprint app launches, web pages, and in-app activities, all without violating Android's permission framework. Our extensive evaluations demonstrate that WinSpy achieves high accuracy (from 70 to 80% detecting website and app launches to over 97% recognizing critical in-app activities). Additionally, we reveal that due to Android's lenient permission management for this mode, window apps can also use Inertial Measurement Unit sensors to launch attacks, such as inferring the user's touch positions outside the window with high precision. Furthermore, we propose systematic mitigations against these vulnerabilities.

Face recognition is essential for identity authentication, but the rich visual clues in facial images pose significant privacy risks, highlighting the critical importance of privacy-preserving solutions. For instance, numerous studies have shown that generative models are capable of effectively performing reconstruction attacks that result in the restoration of original visual clues. To mitigate this threat, we introduce FracFace, a fractal-based privacy-preserving face recognition framework. This approach effectively weakens the visual clues that can be exploited by reconstruction attacks by disrupting the spatial structure in frequency domain features, while retaining the vital visual clues required for identity recognition. To achieve this, we craft a Frequency Channels Refining module that reduces sparsity in the frequency domain. It suppresses visual clues that could be exploited by reconstruction attacks, while preserving features indispensable for recognition, thus making these attacks more challenging. More significantly, we design a Frequency Fractal Mapping module that obfuscates deep representations by remapping refined frequency channels into a fractal-based privacy structure. By leveraging the self-similarity of fractals, this module preserves identity relevant features while enhancing defense capabilities, thereby improving the overall robustness of the protection scheme. Experiments conducted on multiple public face recognition benchmarks demonstrate that the proposed FracFace significantly reduces the visual recoverability of facial features, while maintaining high recognition accuracy, as well as the superiorities over state-of-the-art privacy protection approaches.

The unprecedented image generation capability of text-to-image models makes them double-edged swords. While these models allow users to create exquisite images through simple prompts, they also provide adversaries with opportunities to generate Not-Safe-for-Work (NSFW) content, referred to as the jailbreak attack. Despite built-in safety filters serving as a mitigation, their vulnerabilities and associated safety issues remain a significant concern. In this work, we propose MODX, the first modifier-based attack framework for jailbreaking text-to-image models. Modx leverages a heuristic algorithm with two heuristic functions (constraints) to identify modifiers that adjust the artistic genre to subtly introduce unsafe elements that drive the generated images towards NSFW. This approach takes advantage of the fact that filters are unlikely to reject images in certain styles or artistic forms, effectively inducing the models to generate NSFW content. We demonstrate the feasibility of modifier-based jailbreaking with a theoretical analysis, and provide experimental evidence of the effectiveness of MODX. Our results show that MODX outperforms existing methods in successfully achieving jailbreaking across four state-of-the-art text-to-image models. Moreover, we evaluate MODX across additional NSFW categories and on more models or model versions, demonstrating its strong scalability and generalization. Disclaimer: This paper contains NSFW language and imagery that could be offensive, distressing, and/or upsetting. Reader discretion is advised.

The presence of non-independent and identically distributed (non-IID) data among clients poses a critical challenge to the deployment of Federated Learning (FL) in practice. In response, state-of-the-art solutions known as Clustered Federated Learning (CFL) schemes, such as FL+HC and PACFL, have emerged to tackle this issue. Their main innovation is to cluster non-IID clients into groups of IID clients, such that techniques designated for IID scenarios can be easily applicable. Nonetheless, the robustness of CFL schemes remains largely unexplored, and existing Byzantine-robust defence mechanisms prove inadequate in CFL schemes and non-IID data settings. In this work, we present novel powerful CFL-specific poisoning attacks, named Cluster-U-M and Cluster-U-D. These attacks are designed to significantly reduce the model utility, measured in terms of test accuracy, for benign clients participating in the CFL schemes. Notably, these attacks remain agnostic, requiring no adversarial knowledge regarding defense solutions and benign clients themselves. At a high level, the attacks involve two steps, including cluster poisoning attacks and client-drift exploitation within clusters. The former induces the grouping of clients with different training distributions, and the latter amplifies the difference between each client's optimum and their group's average aggregation. We extensively evaluate the impact of these attacks using FL+HC and PACFL schemes on both small and large scales. The evaluation results demonstrate that the attacks can compromise up to 54% of clients, with a maximum accuracy loss of 48%. Even with only 0.1% clients compromised, which represents a minimal practical adversarial effort, these attacks can still victimize around 4% clients. We evaluate the effectiveness of two state-of-the-art Byzantine-robust defence mechanisms, i.e., FLTrust and FLAME, in countering Cluster-U-M and Cluster-U-D, and find that the attacks can victimize up to 38% of clients with an accuracy loss of 18-38% under the FL+HC scheme.

Websites commonly display cookie banners to inform users about the use and purposes of cookies. However, they may still, whether intentionally or unintentionally (e.g., due to third-party libraries imported), mis-declare cookies that may be abused for tracking. In this work, we introduce COOVER (cookie value examiner) to assess the non-compliance between the website-declared purpose and the semantic-intended purpose of cookies (denoted as potential cookie purpose violation ). We advocate that the value of the cookie is a more reliable indicator of its semantic-intended purpose compared to other features such as expiration time. COOVER decomposes the cookie value into primitive segments representing minimal semantic units, and fine-tunes a GPT-3.5 model to automatically interpret their value-inferred semantics. Based on the interpretation, it classifies cookies into four GDPR-defined purposes. COOVER achieves an F1 score of 95%, significantly outperforming other methods. We employ COOVER to analyze Alexa Top 1k websites to understand the status quo of potential cookie purpose violation on the web. Remarkably, out of 15,339 cookies across these websites, only 3.1% quality as truly necessary cookies, while 44.1% of websites suffer from issues of potential purpose violation.

Large-scale models are typically adapted to meet the diverse requirements of model owners and users.However, maintaining multiple specialized versions of the model is inefficient.In response, we propose Aim, a novel model modulation paradigm that enables a single model to exhibit diverse behaviors to meet the specific end requirements.Aim enables two key modulation modes: utility and focus modulations.The former provides model owners with dynamic control over output quality to deliver varying utility levels, and the latter offers users precise control to shift model's focused input features.Aim introduces a logits redistribution strategy that operates in a training data-agnostic and retraining-free manner.We establish a formal foundation to ensure Aim's regulation capability, based on the statistical properties of logits ordering via joint probability distributions.Our evaluation confirms Aim's practicality and versatility for AI model modulation, with tasks spanning image classification, semantic segmentation and text generation, and prevalent architectures including ResNet, SegFormer and Llama.

OpenAI has enabled third-party developers to build applications around ChatGPT, known as GPTs, to expand its capability to handle complex and specialized tasks. A key feature of GPTs is Retrieval-Augmented Generation (RAG), which allows developers to upload documents containing domain knowledge or application context, referred to as file knowledge. However, these documents often contain sensitive information, and the security mechanisms governing access control in GPTs remains an underexplored area.

Since the inception of Mirai in 2016, a proliferation of advanced botnets targeting Internet of Things (IoT) devices has occurred, resulting in a notable increase in large-scale cyber attacks against online services. The continual emergence of novel strategies characterises the evolving landscape of IoT botnets. Despite this, a comprehensive understanding of this evolving threat remains elusive, impeding the development of robust defence mechanisms. This paper investigated 55 instances of IoT botnets spanning from 2008 to 2021 to elucidate their evolutionary patterns based on prevalent tactics and techniques. A novel taxonomy of IoT botnets is proposed and formulated with attack tactics, techniques, types, and procedures. We augment our existing simulation framework, IoTSecSim, with enhanced functionalities to simulate novel cyber-attack scenarios incorporating diverse network configurations, evolving attack tactics, and defence strategies. Through comprehensive simulations via the extended IoTSecSim, we assessed the impact of these evolving IoT attack tactics and gauged the efficacy of traditional defence mechanisms using various security metrics.

Autonomous Vehicles (AVs) integrate numerous control units , network components, and protocols to operate effectively and interact with their surroundings, such as pedestrians and other vehicles. While these technologies enhance vehicle capabilities and enrich the driving experience, they also introduce new attack surfaces, making AVs vulnerable to cyber-attacks. Such cyber-attacks can lead to severe consequences, including traffic disruption and even threats to human life. Security modelling is crucial to safeguarding AVs as it enables the simulation and analysis of an AV’s security before any potential attacks. However, the existing research on AV security modelling methods for analysing security risks and evaluating the effectiveness of security measures remains limited. In this work, we introduce a novel graphical security model and metrics to assess the security of AV systems. The proposed model utilizes initial network information to build attack graphs and attack trees at different layers of network depth. From this, various metrics are automatically calculated to analyse the security and safety of the AV network. The proposed model is designed to identify potential attack paths, analyse security and safety with precise metrics, and evaluate various defence strategies. We demonstrate the effectiveness of our framework by applying it to two AV networks and distinct AV attack scenarios, showcasing its capability to enhance the security of AVs.

With the proliferation of intelligent healthcare systems, patients’ Personal Health Records (PHR) generated by the Internet of Medical Things (IoMT) in real-time play a vital role in disease diagnosis. The integration of emerging blockchain technologies significantly enhanced the data security inside intelligent medical systems. However, data sharing across different systems based on varied blockchain architectures is still constrained by the unsolved performance and security challenges. This paper constructs a cross-chain data sharing scheme, termed MedExChain, which aims to securely share PHR across heterogeneous blockchain systems. The MedExChain scheme ensures that PHR can be shared across chains even under the performance limitations of IoMT devices. Additionally, the scheme incorporates Cryptographic Reverse Firewall (CRF) and a blockchain audit mechanism to defend against both internal and external security threats. The robustness of our scheme is validated through BAN logic, Scyther tool, Chosen Plaintext Attack (CPA) and Algorithm Substitution Attack (ASA) security analysis verification. Extensive evaluations demonstrate that MedExChain significantly minimizes computation and communication overhead, making it suitable for IoMT devices and fostering the efficient circulation of PHR across diverse blockchain systems.

The wide adoption of deep learning in safety-critical domains has driven the need for formally verifying the robustness of neural networks. A critical challenge in this endeavor lies in addressing the inherent non-linearity of activation functions. The convex hull of the activation function has emerged as a promising solution, as it effectively tightens variable ranges and provides multi-neuron constraints, which together enhance verification precision. Given that constructing exact convex hulls is computationally expensive and even infeasible in most cases, existing research has focused on over-approximating them. Several ad-hoc methods have been devised for specific functions such as ReLU and Sigmoid. Nonetheless, there remains a substantial gap in developing broadly applicable approaches for general activation functions. In this work, we propose WraAct , an approach to efficiently constructing tight over-approximations for activation function hulls. Its core idea is to introduce linear constraints to smooth out the fluctuations in the target function, by leveraging double-linear-piece (DLP) functions to simplify the local geometry. In this way, the problem is reduced to over-approximating DLP functions, which can be efficiently handled. We evaluate WraAct against SBLM+PDDM, the state-of-the-art (SOTA) multi-neuron over-approximation method based on decomposing functions into segments. WraAct outperforms it on commonly-used functions like Sigmoid, Tanh, and MaxPool, offering superior efficiency (average 400X faster) and precision (average 150X) while constructing fewer constraints (average 50% reduction). It can complete the computation of up to 8 input dimensions in 10 seconds. We also integrate WraAct into a neural network verifier to evaluate its capability in verification tasks. On 100 benchmark samples, it significantly enhances the single-neuron verification from under 10 to over 40, and outperforms the multi-neuron verifier PRIMA with up to additional 20 verified samples. On large networks like ResNets with 22k neurons, it can complete the verification of one sample within one minute.

In this paper, we propose p-Blend, an efficient and effective blendshape perturbation mechanism designed to defend against both intra- and cross-app re-identification attacks in virtual reality. p-Blend provides privacy protection when streaming blendshape data to third-party applications on VR devices. In its design, we consider both privacy and utility. p-Blend not only perturbs blendshape values to resist re-identification attacks but also preserves the smoothness of facial animations and the naturalness of facial expressions, ensuring the continued usability of the data. We validate the effectiveness of p-Blend through extensive empirical evaluations and user studies. Quantitative experiments on a large-scale dataset collected from 45 participants demonstrate that p-Blend significantly reduces re-identification accuracy across a range of machine learning models. While pure-random perturbation fails to prevent attacks that exploit statistical features, p-Blend effectively mitigates these risks in both raw and statistical blendshape data. Additionally, user study results show that facial animations generated from p-Blend-perturbed blendshapes maintain greater smoothness and naturalness compared to those using purely random perturbation. The codes and dataset are available at https://github.com/jingwei1016/p-Blend.

With large AI systems and models (LAMs) playing an ever-growing role across diverse applications, their impact on the privacy and cybersecurity of critical infrastructure has become a pressing concern. The LAMPS workshop is dedicated to tackling these emerging challenges, promoting dialogue on cutting-edge developments and ethical issues in safeguarding LAMs within critical infrastructure contexts. Bringing together leading experts from around the world, this workshop will delve into the complex privacy and cybersecurity risks posed by LAMs in critical sectors. Attendees will explore innovative solutions, exchange best practices, and contribute to shaping the future research agenda, emphasizing the crucial balance between advancing AI technologies and securing critical digital and physical infrastructures.

The gradient inversion attack has been demonstrated as a significant privacy threat to federated learning (FL), particularly in continuous domains such as vision models. In contrast, it is often considered less effective or highly dependent on impractical training settings when applied to language models, due to the challenges posed by the discrete nature of tokens in text data. As a result, its potential privacy threats remain largely underestimated, despite FL being an emerging training method for language models. In this work, we propose a domain-specific gradient inversion attack named GRAB (gradient inversion with hybrid optimization). GRAB features two alternating optimization processes to address the challenges caused by practical training settings, including a simultaneous optimization on dropout masks between layers for improved token recovery and a discrete optimization for effective token sequencing. GRAB can recover a significant portion (up to 92.9% recovery rate) of the private training data, outperforming the attack strategy of utilizing discrete optimization with an auxiliary model by notable improvements of up to 28.9% recovery rate in benchmark settings and 48.5% recovery rate in practical settings. GRAB provides a valuable step forward in understanding this privacy threat in the emerging FL training mode of language models.

Generative Adversarial Networks (GANs), as a cornerstone of artificial intelligence (AI), are widely recognized as the intellectual property (IP) of their owners, given the sensitivity of the training data and the commercial value tied to the models. Model extraction attacks, which aim to steal well-trained proprietary models, pose a significant threat to model IP. Nevertheless, current research predominately focuses on the context of machine learning as a service (MLaaS), where the emphasis lies in understanding the attack knowledge acquired through black-box API queries. This restricted perspective exposes a critical gap in investigating model extraction attacks within realistic distributed settings for generative tasks. In this work, we present the first investigation into model extraction attacks against GANs in distributed settings. We provide a comprehensive attack taxonomy, considering three different levels of knowledge the adversary can obtain in practice. Based on it, we introduce a novel model extraction attack named MoEx, which focuses on the GAN-based distributed learning scenario, i.e., Multi-Discriminator GANs, a typical asymmetric distributed setting. MoEx uses the objective function simulation, leveraging data exchanged during the learning process, to approximate the GAN generator owned by the server. We define two attack goals for MoEx, fidelity extraction and accuracy extraction . Then we comprehensively evaluate the effectiveness of MoEx's two goals with real-world datasets. Our results demonstrate its robust capabilities in extracting generators with high fidelity and accuracy compared with existing methods.

Virtual personal assistants (VPA) services encompass a large number of third-party applications (or apps) to enrich their functionalities. These apps have been well examined to scrutinize their data collection behaviors against their declared privacy policies. Nonetheless, it is often overlooked that most users tend to ignore privacy policies at the installation time. Dishonest developers thus can exploit this situation by embedding excessive declarations to cover their data collection behaviors during compliance auditing.

Multi-channel distribution of Android apps offers convenience to users, yet simultaneously introduces security concerns. Although apps published on Google Play and third-party markets share the same version code, differences in app content may still arise. Notably, a recent incident involving the third-party market version of Pinduoduo app containing malicious code highlights the intentionally-differentiated implementations of app functionalities by developers between Google Play and third-party markets. The case of Pinduoduo may be just the tip of the iceberg, underscoring the need for a comprehensive investigation of the disparities between Google Play and third-party market versions of apps.In this work, we systematically analyze the differences in security and privacy of cross-market apps that claim to share the same version code. Specifically, we propose three research questions that cover differences in app protection, security threats, and permission usage. To answer these questions, we constructed a dataset containing 17,218 app pairs (filtered from 236,731 apps) and permission mappings (27,046 SDK mappings, 1,656 ContentProvider mappings, and 309 Intent mappings) for API levels 16 - 33. This dataset enables us to perform a comprehensive differential analysis. Consequently, our investigation unveiled a series of captivating and insightful findings. Approximately 29.02% of apps show differences in one or all three aspects. For example, the third-party market versions of apps often request more permissions compared to their Google Play counterparts, particularly among apps in the game category. Our work can help developers and app store operators improve cross-market app consistency, enhancing the quality of the Android app ecosystem and user experience.

Model optimization, such as pruning and quantization, has become the de facto pre-deployment phase when deploying deep learning (DL) models on resource-constrained platforms. However, the complexity of DL models often leads to non-trivial bugs in model optimizers, known as model optimization bugs (MOBs). These MOBs are characterized by involving complex data types and layer structures inherent to DL models, causing significant hurdles in detecting them through traditional static analysis and dynamic testing techniques. In this work, we leverage Large Language Models (LLMs) with prompting techniques to generate test cases for MOB detection. We explore how LLMs can draw an understanding of the MOB domain from scattered bug instances and generalize to detect new ones, a paradigm we term as concentration and diffusion. We extract MOB domain knowledge from the artifacts of known MOBs, such as their issue reports and fixes, and design knowledge-aware prompts to guide LLMs in generating effective test cases. The domain knowledge of code structure and error description provides precise in-depth depictions of the problem domain, i.e., the concentration, and heuristic directions to generate innovative test cases, i.e., the diffusion. Our approach is implemented as a tool named YanHui and benchmarked against existing few-shot LLM-based fuzzing techniques. Test cases generated by YanHui demonstrate enhanced capability to find relevant API and data combinations for exposing MOBs, leading to an 11.4% increase in generating syntactically valid code and a 22.3% increase in generating on-target code specific to model optimization. YanHui detects 17 MOBs, and among them, five are deep MOBs that are difficult to reveal without our prompting technique.

ChatGPT has enabled third-party developers to create plugins to expand ChatGPT's capabilities. These plugins are distributed through OpenAI's plugin store, making them easily accessible to users. With ChatGPT as the backbone, this app ecosystem has illustrated great business potential by offering users personalized services in a conversational manner. Nonetheless, many crucial aspects regarding app development, deployment, and security of this ecosystem have yet to be thoroughly studied in the research community, potentially hindering a broader adoption by both developers and users. In this work, we conduct the first comprehensive study of the ChatGPT app ecosystem, aiming to illuminate its landscape for our research community. Our study examines the distribution and deployment models in the integration of LLMs and third-party apps, and assesses their security and privacy implications. We uncover an uneven distribution of functionality among ChatGPT plugins, highlighting prevalent and emerging topics. We also identify severe flaws in the authentication and user data protection for third-party app APIs integrated within LLMs, revealing a concerning status quo of security and privacy in this app ecosystem. Our work provides insights for the secure and sustainable development of this rapidly evolving ecosystem.

Vision-language pre-trained (VLP) models have been the foundation of numerous vision-language tasks. Given their prevalence, it becomes imperative to assess their adversarial robustness, especially when deploying them in security-crucial real-world applications. Traditionally, adversarial perturbations generated for this assessment target specific VLP models, datasets, and/or downstream tasks. This practice suffers from low transferability and additional computation costs when transitioning to new scenarios.

The growing complexity of deep neural network models in modern application domains necessitates a complex training process that involves extensive data, sophisticated design, and substantial computation. The trained model inherently encapsulates the intellectual property owned by the model developer (or the model owner). Consequently, safeguarding the model from unauthorized use by entities who obtain access to the model (or the model controllers), i.e., preserving the fundamental rights and proprietary interests of the model owner, has become a critical necessity.In this work, we propose CORELOCKER, employing the strategic extraction of a small subset of significant weights from the neural network. This subset serves as the access key to unlock the model’s complete capability. The extraction of the key can be customized to varying levels of utility that the model owner intends to release. Authorized users with the access key have full access to the model, while unauthorized users can have access to only part of its capability. We establish a formal foundation to underpin CORELOCKER, which provides crucial lower and upper bounds for the utility disparity between pre- and post-protected networks. We evaluate CORELOCKER using representative datasets such as Fashion-MNIST, CIFAR-10, and CIFAR-100, as well as real-world models including Vg-gNet, ResNet, and DenseNet. Our experimental results confirm its efficacy. We also demonstrate CORELOCKER’s resilience against advanced model restoration attacks based on fine-tuning and pruning.

Since browser extensions are prevailingly executed in the background to enable extra functionalities and enhance the user experience for web browsers, the potential over-collection of personal data beyond the necessity for given purposes is always ignored by ordinary users. Existing privacy regulations, such as the principle of Data Minimization in GDPR, have provided the criteria that only directly relevant and necessary data for specified purposes should be collected. Various tools have made efforts to examine the compliance of data minimization and its equivalent in different application domains. To our knowledge, in the area of browser extensions, there is still a gap between the general data minimization principle and precisely defined extension behaviors. We propose MINDAExT, a framework that takes one step further to automatically examine end-to-end data minimization practices in browser extensions by description text analysis and hybrid program analysis techniques. In our large-scale measurement, covering around 200K extensions collected in October 2023, we find that 38.0 % of extensions are likely to collect private user data outside their essential functionality scopes. They are distributed across all categories, exhibiting distinct patterns of the target data types. Our evaluation shows that MINDAEXT can detect the data over-collection with a precision of 74.3 %.

Web-based trigger-action platforms (TAP) allow users to integrate Internet of Things (IoT) systems and online services into trigger-action integrations (TAIs), facilitating rich automation tasks known as applets. Despite their benefits, these integrations~(typically involving the TAP, trigger, and action service providers) pose significant security and privacy challenges, such as mis-triggering and data leakage. This work investigates cross-entity permission management within TAIs to address the underlying causes of these security and privacy issues, emphasizing permission-functionality consistency to ensure fairness in permission requests. We introduce PFCon, a system that leverages GPT-based language models for analyzing required and requested permissions, revealing excessive permission requests in a large-scale study of IFTTT TAP. Our findings highlight the need for service providers to enforce permission-functionality consistency, raising awareness of the importance of security and privacy in TAI.

The increasing demand for remote work and virtual interactions has heightened the usage of business collaboration platforms~(BCPs), with Google Workspace as a prominent example. These platforms enhance team collaboration by integrating Google Docs, Slides, Calendar, and feature-rich third-party applications (add-ons). However, such integration of multiple users and entities has inadvertently introduced new and complex attack surfaces, elevating security and privacy risks in resource management to unprecedented levels. In this study, we conduct a systematic study on the effectiveness of the cross-entity resource management in Google Workspace, the most popular BCP. Our study unveils the access control enforcement in real-world BCPs for the first time. Based on this, we formulate the attack surfaces inherent in BCPs and conduct a comprehensive assessment, pinpointing three vulnerability types leading to distinct attacks. An analysis of 4,732 marketplace add-ons reveals that approximately 70% are potentially vulnerable to these attacks. We propose robust countermeasures to improve BCP security, urging immediate action and setting a foundation for future research.

The energy industry is undergoing significant transformations as it strives to achieve net-zero emissions and future-proof its infrastructure, where every participant in the power grid has the potential to both consume and produce energy resources. Federated learning -- which enables multiple participants to collaboratively train a model without aggregating the training data -- becomes a viable technology. However, the global model parameters that have to be shared for optimization are still susceptible to training data leakage. In this work, we propose confined gradient descent (CGD) that enhances the privacy of federated learning by eliminating the sharing of global model parameters. CGD exploits the fact that a gradient descent optimization can start with a set of discrete points and converges to another set in the neighborhood of the global minimum of the objective function. As such, each participant can independently initiate its own private global model~(referred to as the confined model ), and collaboratively learn it towards the optimum. The updates to their own models are worked out in a secure collaborative way during the training process.In such a manner, CGD retains the ability of learning from distributed data but greatly diminishes information sharing. Such a strategy also allows the proprietary confined models to adapt to the heterogeneity in federated learning, providing inherent benefits of fairness. We theoretically and empirically demonstrate that decentralized CGD øne provides a stronger differential privacy (DP) protection; \two is robust against the state-of-the-art poisoning privacy attacks; þree results in bounded fairness guarantee among participants; and \four provides high test accuracy (comparable with centralized learning) with a bounded convergence rate over four real-world datasets.

The proliferation of the Internet of Things (IoT) devices has provided attackers with tremendous opportunities to launch various cyber-attacks. It has been challenging to analyse the impact of cyber-attacks and evaluate the effectiveness of defences in real IoT environments due to the scale and heterogeneity of IoT networks. In this work, we propose a novel simulation framework and a software tool, IoT Security Simulator (IoTSecSim). IoTSecSim is operated based on a framework we propose for modelling and simulating cyber-attacks and various defences in IoT networks. IoTSecSim is not only able to support the creation of an IoT network with flexible settings of IoT devices and topology information but also models the attack behaviours, node-level, and network-level defences. Moreover, a systematic security evaluation can be performed by comparing the results based on the calculation of security metrics. We perform simulations with case studies on Mirai malware and its variants to model cyber-attack behaviours on IoT networks and evaluate the impact of these attacks and the effectiveness of defence techniques via IoTSecSim. Then, we carry out a sensitivity analysis to justify that the simulation results produced by IoTSecSim are accurate and feasible when compared with related works. We also perform a comparative performance analysis with four combinations of cyber-attack behaviours and show that these behaviours can influence IoT malware propagation in different situations. We consider multiple attacker models and deploy conventional defence techniques (including firewall, intrusion detection, and vulnerability patching) to investigate the effectiveness of defence techniques. IoTSecSim provides a generalised and extensible simulation framework that enables users to model emerging cyber-attacks against IoT networks and evaluate the effectiveness of defences against these attacks. This helps users to focus on the design and performance evaluation of new defences before the actual implementation and deployment of the defences are required.

Cyber-physical infrastructure faces threats from evasive false data injection attacks that can significantly impact their security and performance. Adversarial attacks are a popular evasive false data injection threat model that generally targets AI/ML models employed in cyber-physical systems (CPSs). However, AI/ML models are often not necessary to carry out several key functions of critical CPSs. In this study, we explore the potential for designing effective adversarial attacks targeting critical functions within cyber-physical infrastructure, that do not rely on AI systems, while retaining their adversarial nature against related black-box AI and non-AI functionalities. Specifically, we introduce an evasive deep black-box adversarial attack (DeeBBAA) designed to disrupt the nonlinear state estimation process of an unidentified power network. The attack is engineered to operate under practical constraints, without prior knowledge of the network’s topology, and by targeting fewer than twenty-five percent of the network’s measurements during data collection (or eavesdropping) and the subsequent online attack injection phases. The study demonstrates that even within these restrictive parameters, DeeBBAA exhibits significant evasiveness against a broad spectrum of conventional, statistical, and machine learning-based cyberattack detection techniques, resulting in substantial deviations in the estimated network states.

Convex hulls are commonly used to tackle the non-linearity of activation functions in the verification of neural networks. Computing the exact convex hull is a costly task though. In this work, we propose a fast and precise approach to over-approximating the convex hull of the ReLU function (referred to as the ReLU hull ), one of the most used activation functions. Our key insight is to formulate a convex polytope that “wraps” the ReLU hull, by reusing the linear pieces of the ReLU function as the lower faces and constructing upper faces that are adjacent to the lower faces. The upper faces can be efficiently constructed based on the edges and vertices of the lower faces, given that an n -dimensional (or simply n d hereafter) hyperplane can be determined by an ( n - 1 ) d hyperplane and a point outside of it. We implement our approach as WraLU , and evaluate its performance in terms of precision, efficiency, constraint complexity, and scalability. WraLU outperforms existing advanced methods by generating fewer constraints to achieve tighter approximation in less time. It exhibits versatility by effectively addressing arbitrary input polytopes and higher-dimensional cases, which are beyond the capabilities of existing methods. We integrate WraLU into PRIMA, a state-of-the-art neural network verifier, and apply it to verify large-scale ReLU-based neural networks. Our experimental results demonstrate that WraLU achieves a high efficiency without compromising precision. It reduces the number of constraints that need to be solved by the linear programming solver by up to half, while delivering comparable or even superior results compared to the state-of-the-art verifiers.

An app ecosystem built around virtual personal assistant (VPA) services becomes flourishing in recent years, fueled by the booming of the Internet of Things (IoT). A large number of functionality-rich VPA applications (or apps for short) have been released through app stores, and become easily-accessible by users through their smart speakers. In response to the increasingly stringent data protection regulations around the world, VPA service providers require app developers to include a privacy policy that declares their data handling practices. These privacy policies serve as the de facto agreement between developers and users, and may be taken as the basis in resolving conflicts in the event of a data breach. Therefore, it is essential that privacy policy documents are crafted in a clear, easy-to-understand, and unambiguous way. In this work, we conduct the first systematic study on the quality of privacy policies in the VPA app domain. Based on our review of literature and documents from standard working groups, we identify four metrics that enable the quality of the privacy policy to become measurable, including timeliness, availability, completeness and readability. We then develop QuPer, which extracts the meta features (e.g., update history) and linguistic features (e.g., sentence semantics) from privacy policies, and assesses their quality. Our analysis reveals that the status of the quality of privacy policies in the VPA app domain is concerning. For instance, only 1.17% of privacy policies completely cover all contents that are regarded as privacy concerns by legislation (e.g., GDPR article 13) and relevant literature. Our findings are expected to raise an alert among the VPA app developers and provide them with guidelines for creating high-quality privacy policy documents. We also encourage app store operators to implement a vetting process that ensures the quality of privacy policies before apps are released to the public.

The collaborative nature of federated learning (FL) poses a major threat in the form of manipulation of local training data and local updates, known as the Byzantine poisoning attack. To address this issue, many Byzantine-robust aggregation rules (AGRs) have been proposed to filter out or moderate suspicious local updates uploaded by Byzantine participants. This paper introduces a novel approach called AGRAMPLIFIER, aiming to simultaneously improve robustness, fidelity, and efficiency of the existingAGRs. The core idea of AGRAMPLIFIER is to amplify the “morality” of local updates by identifying the most repressive features of each gradient update, which provides a clearer distinction between malicious and benign updates, consequently improving the detection effect. To achieve this objective, two approaches, namelyAGRMPandAGRXAI, are proposed.AGRMPorganizes local updates into patches and extracts the largest value from each patch, whileAGRXAIleverages explainable AI methods to extract the gradient of the most activated features. By equipping AGRAMPLIFIER with the existing Byzantine-robust mechanisms, we successfully enhance the model robustness, maintaining its fidelity and improving overall efficiency. AGRAMPLIFIER is universally compatible with the existing Byzantine-robust mechanisms. The paper demonstrates its effectiveness by integrating it with all mainstreamAGRmechanisms. Extensive evaluations conducted on seven datasets from diverse domains against seven representative poisoning attacks consistently show enhancements in robustness, fidelity, and efficiency, with average gains of 40.08%, 39.18%, and 10.68%, respectively.

Corruptions due to data perturbations and label noise are prevalent in the datasets from unreliable sources, which poses significant threats to model training. Despite existing efforts in developing robust models, current learning methods commonly overlook the possible co-existence of both corruptions, limiting the effectiveness and practicability of the model. In this paper, we develop an Effective and Robust Adversarial Training (ERAT) framework to simultaneously handle two types of corruption (i.e., data and label) without prior knowledge of their specifics. We propose a hybrid adversarial training surrounding multiple potential adversarial perturbations, alongside a semi-supervised learning based on class-rebalancing sample selection to enhance the resilience of the model for dual corruption. On the one hand, in the proposed adversarial training, the perturbation generation module learns multiple surrogate malicious data perturbations by taking a DNN model as the victim, while the model is trained to maintain semantic consistency between the original data and the hybrid perturbed data. It is expected to enable the model to cope with unpredictable perturbations in real-world data corruption. On the other hand, a class-rebalancing data selection strategy is designed to fairly differentiate clean labels from noisy labels. Semi-supervised learning is performed accordingly by discarding noisy labels. Extensive experiments demonstrate the superiority of the proposed ERAT framework.

Vulnerability detectors based on deep learning (DL) models have proven their effectiveness in recent years. However, the shroud of opacity surrounding the decision-making process of these detectors makes it difficult for security analysts to comprehend. To address this, various explanation approaches have been proposed to explain the predictions by highlighting important features, which have been demonstrated effective in domains such as computer vision and natural language processing. Unfortunately, there is still a lack of in-depth evaluation of vulnerability-critical features, such as fine-grained vulnerability-related code lines, learned and understood by these explanation approaches. In this study, we first evaluate the performance of ten explanation approaches for vulnerability detectors based on graph and sequence representations, measured by two quantitative metrics including fidelity and vulnerability line coverage rate. Our results show that fidelity alone is insufficent for evaluating these approaches, as fidelity incurs significant fluctuations across different datasets and detectors. We subsequently check the precision of the vulnerability-related code lines reported by the explanation approaches, and find poor accuracy in this task among all of them. This can be attributed to the inefficiency of explainers in selecting important features and the presence of irrelevant artifacts learned by DL-based detectors.

The increasingly popular virtual personal assistant (VPA) services, e.g., Amazon Alexa and Google Assistant, enable third-party developers to create and release VPA apps for end users to access through smart speakers. Given that VPA apps handle sensitive personal data, VPA service providers require developers to release a privacy policy document to declare their data handling practice. The privacy policies are regarded as legal or semi-legal documents, which are usually lengthy and complex for users to understand. In this work, we conducted a subjective study to investigate the level of users’ understanding of the privacy policies, targeting the VPA apps (i.e., skills) of Amazon Alexa, the most popular VPA service. Our study focused on technical terms, one of the greatest hurdles to users’ understanding. We found that 84.2% of our participants faced difficulty in understanding technical terms appeared in the skills’ privacy policies, even for participants with IT background. Additionally, 64.3% of them reported that explanations for the technical terms are generally lacking. To address this issue, we proposed two principles, i.e., domain-specificity principle and implication-oriented principle, to guide skill developers in creating easy-to-understand privacy policies. We evaluated their effectiveness by creating explanation sentences for 23 representative terms and examining users’ understanding through a second user study. Our results show that using explanation sentences based on these principles can significantly improve users’ understanding.

Federated learning (FL) is a widely used distributed machine learning framework. However, recent studies have shown its susceptibility to poisoning membership inference attacks (MIA). In MIA, adversaries maliciously manipulate the local updates on selected samples and share the gradients with the server (i.e., poisoning). Since honest clients perform gradient descent on samples locally, an adversary can distinguish whether the attacked sample is a training sample based on observation of the change of the sample’s prediction. This type of attack exacerbates traditional passive MIA, yet the defense mechanisms remain largely unexplored.

Although machine learning models achieve high classification accuracy against benign examples, they are vulnerable to adversarial machine learning (AML) attacks which generate adversarial examples by adding well-crafted perturbations to the benign examples. The perturbations can be increased to enhance the attack success rate, however, if the perturbations are added without considering the semantic or perceptual similarity between the benign and adversarial examples, the attack can be easily perceived/detected. As such, there exists a trade-off between the attack success rate and the perceptual similarity. In this paper, we propose a novel Semantic-Preserving Adversarial Transformation (SPAT) framework which facilitates an advantageous trade-off between the two metrics. SPAT modifies the optimisation objective of an AML attack to include the goal of increasing the attack success rate as well as the goal of maintaining the perceptual similarity between benign and adversarial examples. Our experiments on a variety of datasets including CIFAR-10, GTSRB, and MNIST demonstrate that SPAT-transformed AML attacks achieve better perceptual similarity while maintaining the attack success rates as the conventional AML attacks.

When deploying pre-trained neural network models in real-world applications, model consumers often encounter resource-constraint platforms such as mobile and smart devices. They typically use the pruning technique to reduce the size and complexity of the model, generating a lighter one with less resource consumption. Nonetheless, most existing pruning methods are proposed with a premise that the model after being pruned has a chance to be fine-tuned or even retrained based on the original training data. This may be unrealistic in practice, as the data controllers are often reluctant to provide their model consumers with the original data. In this work, we study the neural network pruning in the data-free context, aiming to yield lightweight models that are not only accurate in prediction but also robust against undesired inputs in open-world deployments. Considering the absence of fine-tuning and retraining that can fix the mis-pruned units, we replace the traditional aggressive one-shot strategy with a conservative one that treats model pruning as a progressive process. We propose a pruning method based on stochastic optimization that uses robustness-related metrics to guide the pruning process. Our method is evaluated with a series of experiments on diverse neural network models. The experimental results show that it significantly outperforms existing one-shot data-free pruning approaches in terms of robustness preservation and accuracy.

Face recognition is widely adopted in our daily life in recent years. It usually relies on sophisticated techniques to achieve high accuracy in identifying or verifying the identities of given face images. Artificial intelligence (AI), especially deep learning, is a popular technique used in face recognition due to its high accuracy, known as the deep face recognition. However, the reliability of the deep face recognition models becomes a concern, especially in security-critical applications. The main challenge is the "black-box" nature of the sophisticated internal structure of the models. Explainable AI has emerged as a solution that provides meaningful explanations to help humans understand the complicated internal structure of the deep learning models, and increase the transparency and interpretability of the "black box". However, this is often at the cost of model accuracy. In this paper, we propose an approach to increasing both the accuracy and interpretation of quantitatively explainable AI models for face recognition. It increases the accuracy of the explainable face recognition models by applying improved loss functions and enhances quantitative interpretability by adding a new visualisation feature. The proposed approach is validated using advanced deep face recognition models and is compared with existing approaches to demonstrate its better performance.

Due to the great advance in machine learning (ML) techniques, numerous ML models are expanding their application domains in recent years. To adapt for resource-constrained platforms such as mobile and Internet of Things (IoT) devices, pre-trained models are often processed to enhance their efficiency and compactness, using optimization techniques such as pruning and quantization. Similar to the optimization process in other complex systems, e.g., program compilers and databases, optimizations for ML models can contain bugs, leading to severe consequences such as system crashes and financial loss. While bugs in training, compiling and deployment stages have been extensively studied, there is still a lack of systematic understanding and characterization of model optimization bugs (MOBs). In this work, we conduct the first empirical study to identify and characterize MOBs. We collect a comprehensive dataset containing 371 MOBs from TensorFlow and PyTorch, the most extensively used open-source ML frameworks, covering the entire development time span of their optimizers (May 2019 to August 2022). We then investigate the collected bugs from various perspectives, including their symptoms, root causes, life cycles, detection and fixes. Our work unveils the status quo of MOBs in the wild, and reveals their features on which future detection techniques can be based. Our findings also serve as a warning to the developers and the users of ML frameworks, and an appeal to our research community to enact dedicated countermeasures.

Internet of Things (IoT) has become prevalent in various fields, especially in the context of home automation (HA). To better control HA-IoT devices, especially to integrate several devices for rich smart functionalities, trigger-action programming, such as the If This Then That (IFTTT), has become a popular paradigm. Leveraging it, novice users can easily specify their intent in applets regarding how to control a device/service through another once a specific condition is met. Nevertheless, the users may design IFTTT-style integrations inappropriately, due to lack of security experience or unawareness of the security impact of cyber-attacks against individual devices. This has caused financial loss, privacy leakage, unauthorized access and other security issues. To address these problems, this work proposes a systematic framework named MEDIC to model smart home integrations and check their security. It automatically generates models incorporating the service/device behaviors and action rules of the applets, while taking into consideration the external attacks and in-device vulnerabilities. Our approach takes around one second to complete the modeling and checking of one integration. We carried out experiments based on 200 integrations created from a user study and a dataset crawled from ifttt.com. To our great surprise, nearly 83% of these integrations have security issues.

Mini-programs (MiniApps), lightweight versions of full-featured mobile apps that run inside a host app such as WeChat, have become increasingly popular due to their simplified and convenient user experiences. However, MiniApps raise new security and privacy concerns as they can access partially or all of host apps' system resources, including sensitive personal data. While taint detection has been proven effective in addressing this kind of concerns, existing taint detection techniques for mobile apps cannot be directly applied to MiniApps. The main reason is that the key logics of MiniApps are usually written in J avaScript, and its intrinsic characteristics (function-level scope, dynamic types, synchronous programming, and code obfuscation) prevent existing taint detection techniques from precisely propagating the taints. To address this problem, we propose a novel taint detection technique, Wemint, that detects sensitive information leaks in MiniApps. Specifically, Wemint facilitates taint propagation via building a context-based model based on the operational prin-ciple of MiniApps and J avaScript, and addresses asynchronous function calls by modeling their callbacks explicitly in taint rules. In addition, due to the adoption of Abstract Syntax Trees (ASTs) for code representation during taint detection, Wemint exhibits better robustness against the commonly-applied code obfuscation. Our experimental results show that Wemint can effectively detect sensitive information leaks in WeChat MiniApps, as well as trace the path of sensitive data flows. By applying Wemint to over 20K suspicious MiniApps, we found that over 7.5K (36.5 %) of them have sensitive data leaks, and Wemint outperforms the state-of-the-art DoubleX based techniques in detecting these leaks.

Recently, several abstraction refinement techniques have been proposed to improve the verification precision for deep neural networks (DNNs). However, these techniques usually take many refinement steps to verify a property and the refinement decision in each step is hard to interpret, thus hindering their analysis, reasoning and optimization.In this work, we propose SURGEON, a novel DNN verification refinement approach that is both effective and interpretable, allowing analyst to understand why and how each refinement decision is made. The main insight is to leverage the ‘interpretable’ nature of debugging processes and formulate the verification refinement problem as a debugging problem. Given a failed verification procedure, SURGEON refines it in an iterative manner and, in each iteration, it effectively identifies the root cause of the failure and heuristically generates fixes according to abstract transformers.We have implemented SURGEON in a prototype and evaluated it using a set of local robustness verification problems. Besides the interpretability, the experimental results show our approach can improve the precision of base verification methods and is more effective than existing refinement techniques.

The Poisoning Membership Inference Attack (PMIA) is a newly emerging privacy attack that poses a significant threat to federated learning (FL). An adversary conducts data poisoning (i.e., performing adversarial manipulations on training examples) to extract membership information by exploiting the changes in loss resulting from data poisoning. The PMIA significantly exacerbates the traditional poisoning attack that is primarily focused on model corruption. However, there has been a lack of a comprehensive systematic study that thoroughly investigates this topic. In this work, we conduct a benchmark evaluation to assess the performance of PMIA against the Byzantine-robust FL setting that is specifically designed to mitigate poisoning attacks. We find that all existing coordinate-wise averaging mechanisms fail to defend against the PMIA, while the detect-then-drop strategy was proven to be effective in most cases, implying that the poison injection is memorized and the poisonous effect rarely dissipates. Inspired by this observation, we propose AgrEvader, a PMIA that maximizes the adversarial impact on the victim samples while circumventing the detection by Byzantine-robust mechanisms. AgrEvader significantly outperforms existing PMIAs. For instance, AgrEvader achieved a high attack accuracy of between 72.78% (on CIFAR-10) to 97.80% (on Texas100), which is an average accuracy increase of 13.89% compared to the strongest PMIA reported in the literature. We evaluated AgrEvader on five datasets across different domains, against a comprehensive list of threat models, which included black-box, gray-box and white-box models for targeted and non-targeted scenarios. AgrEvader demonstrated consistent high accuracy across all settings tested. The code is available at: https://github.com/PrivSecML/AgrEvader.

Genome-wide analysis has demonstrated both health and social benefits. However, large scale sharing of such data may reveal sensitive information about individuals. One of the emerging challenges is identity tracing attack that exploits correlations among genomic data to reveal the identity of DNA samples. In this paper, we first demonstrate that the adversary can narrow down the sample's identity by detecting his/her genetic relatives and quantify such privacy threat by employing a Shannon entropy-based measurement. For example, we exemplify that when the dataset size reaches 30% of the population, for any target from that population, the uncertainty of the target's identity is reduced to merely 2.3 bits of entropy (i.e., the identity is pinned down within 5 people). Direct application of existing approaches such as differential privacy (DP), secure multiparty computation (MPC) and homomorphic encryption (HE) may not be applicable to this challenge in genome-wide analysis because of the compromise on utility (i.e., accuracy or efficiency). Towards addressing this challenge, this paper proposes a framework named$\upsilon$Fragto facilitate privacy-preserving data sharing and computation in genome-wide analysis.$\upsilon$Fragmitigates privacy risks by using a vertical fragmentation to disrupt the genetic architecture on which the adversary relies for identity tracing without sacrificing the capability of genome-wide analysis. We theoretically prove that it preserves the correctness of the primitive functionalities and algorithms ranging from basic summary statistics to advanced neural networks. Our experiments demonstrate that$\upsilon$Fragoutperforms secure multiparty computation (MPC) and homomorphic encryption (HE) protocols, with a speedup of more than 221x for training neural networks, and also traditional non-private algorithms and a state-of-the-art noise-based differential privacy (DP) solution in most settings.

Deep cross-modal retrieval techniques have recently achieved remarkable performance, which also poses severe threats to data privacy potentially. Nowadays, enormous user-generated contents that convey personal information are released and shared on the Internet. One may abuse a retrieval system to pinpoint sensitive information of a particular Internet user, causing privacy leakage. In this article, we propose a data-centric Proactive Privacy-preserving Cross-modal Learning algorithm that fulfills the protection purpose by employing a generator to transform original data into adversarial data with quasi-imperceptible perturbations before releasing them. When the data source is infiltrated, the inside adversarial data can confuse retrieval models under the attacker’s control to make erroneous predictions. We consider the protection under a realistic and challenging setting where the prior knowledge of malicious models is agnostic. To handle this, a surrogate retrieval model is instead introduced, acting as the target to fool. The whole network is trained under a game-theoretical framework, where the generator and the retrieval model persistently evolve to fight against each other. To facilitate the optimization, a Gradient Reversal Layer module is inserted between two models, enabling a one-step learning fashion. Extensive experiments on widely used realistic datasets prove the effectiveness of the proposed method.

Manipulation of local training data and local updates, i.e., the Byzantine poisoning attack, is the main threat arising from the collaborative nature of the federated learning (FL) paradigm. Many Byzantine-robust aggregation algorithms (AGRs) have been proposed to filter out or moderate suspicious local updates uploaded by Byzantine participants at the central aggregator. However, they largely suffer from model quality degradation due to the over-removal of local updates or/and the inefficiency caused by the expensive analysis of the high-dimensional local updates.

Virtual personal assistant (VPA) services, e.g. Amazon Alexa and Google Assistant, are becoming increasingly popular recently. Users interact with them through voice-based apps, e.g. Amazon Alexa skills and Google Assistant actions. Unlike the desktop and mobile apps which have visible and intuitive graphical user interface (GUI) to facilitate interaction, VPA apps convey information purely verbally through the voice user interface (VUI), which is known to be limited in its invisibility, single mode and high demand of user attention. This may lead to various problems on the usability and correctness of VPA apps.

Browser extensions have emerged as integrated characteristics in modern browsers, with the aim to boost the online browsing experience. Their advantageous position between a user and the Internet endows them with easy access to the user’s sensitive data, which has raised mounting privacy concerns from both legislators and extension users. In this work, we propose an end-to-end approach to automatically diagnosing the privacy compliance violations among extensions. It analyzes the compliance of privacy policy versus regulation requirements and their actual privacy-related practices during runtime. This approach can serve the extension users, developers and store operators as an efficient and practical detection mechanism for privacy compliance violations.

A large number of functionality-rich and easily accessible applications have become popular among various virtual personal assistant (VPA) services such as Amazon Alexa. VPA applications (or VPA apps for short) are accompanied by a privacy policy document that informs users of their data handling practices. These documents are usually lengthy and complex for users to comprehend, and developers may intentionally or unintentionally fail to comply with them. In this work, we conduct the first systematic study on the privacy policy compliance issue of VPA apps. We develop Skipper, which targets Amazon Alexa skills. It automatically depicts the skill into the declared privacy profile by analyzing their privacy policy documents with Natural Language Processing (NLP) and machine learning techniques, and derives the behavioral privacy profile of the skill through a black-box testing. We conduct a large-scale analysis on all skills listed on Alexa store, and find that a large number of skills suffer from the privacy policy noncompliance issues.

Data owners have the right to request for deleting their data from a machine learning (ML) model. In response, a naïve way is to retrain the model with the original dataset excluding the data to forget, which is however unrealistic as the required dataset may no longer be available and the retraining process is usually computationally expensive. To cope with this reality, machine unlearning has recently attained much attention, which aims to enable data removal from a trained ML model responding to deletion requests, without retraining the model from scratch or full access to the original training dataset. Existing unlearning methods mainly focus on handling conventional ML methods, while unlearning deep neural networks (DNNs) based models remains underexplored, especially for the ones trained on large-scale datasets.

Data annotations obtained for supervised learning often suffer from label noise, which would inevitably incur unreliable deep neural networks. Existing solutions to this problem typically limit the scope to instance-independent label noise. Due to the high illegibility of data and the inexperience of annotators, instance-dependent noise has also been widely observed, however, not being investigated. In this paper, we propose a novel \underlineIDE ntify and \underlineAL ign (IDEAL) methodology, which aims to eliminate the feature distribution shift raised by a broad spectrum of noise patterns. The proposed model is capable of learning noise-resilient feature representations, thereby correctly predicting data instances. More specifically, we formulate the robust learning against noisy labels as a domain adaptation problem by identifying noisy data (i.e., data samples with incorrect labels) and clean data from the dataset as two domains and minimizing their domain discrepancy in the feature space. In this framework, a high-order-ensemble adaptation network is devised to provide high-confidence predictions, according to which a specific criterion is defined for differentiating clean and noisy data. A new metric based on data augmentation is designed to measure the discrepancy between the clean and noisy domains. Along with a min-max learning strategy between the feature encoder and the classifier on the discrepancy, the domain gap will be bridged, which encourages a noise-resilient model. In-depth theoretical analysis and extensive experiments on widely-used benchmark datasets demonstrate the effectiveness of the proposed method.

WPA (Wi-Fi Protected Access) Enterprise is the de facto standard for safeguarding enterprise-level wireless networks. It relies on Transport Layer Security (TLS) to establish a secure tunnel during its authentication process, and thus the notoriously error-prone certificate validation may haunt it. Incorrect validation may lead to the SSL/TLS man-in-the-middle attack, or the evil twin attack in the context of wireless networking, where the supplicant connects and unwittingly sends authentication credentials to a fake access point.

Ever since the beginning of the outbreak of the COVID-19 pandemic, researchers from interdisciplinary domains have worked together to fight against the crisis. The open source community, plays a vital role in coping with the pandemic which is inherently a collaborative process. Plenty of COVID-19 related datasets, tools, software, deep learning models, are created and shared in research communities with great efforts. However, COVID-19 themed open source projects have not been systematically studied, and we are still unaware how the open source community helps combat COVID-19 in practice. To fill this void, in this paper, we take the first step to study COVID-19 themed repositories in GitHub, one of the most popular collaborative platforms. We have collected over 67K COVID-19 themed GitHub repositories till July 2020. We then characterize them from a number of aspects and classify them into six categories. We further investigate the contribution patterns of the contributors, and development and maintenance patterns of the repositories. This study sheds light on the promising direction of adopting open source technologies and resources to rapidly tackle the worldwide public health emergency in practice, and reveals existing challenges for improvement.

With many trigger-action platforms that integrate Internet of Things (IoT) systems and online services, rich functionalities transparently connecting digital and physical worlds become easily accessible for the end users. On the other hand, such facilities incorporate multiple parties whose data control policies may radically differ and even contradict each other, and thus privacy violations may arise throughout the lifecycle (e.g., generation and transmission) of triggers and actions. In this work, we conduct an in-depth study on the privacy issues in multi-party trigger-action integration platforms (TAIPs). We first characterize privacy violations that may arise with the integration of heterogeneous systems and services. Based on this knowledge, we propose Taifu, a dynamic testing approach to identify privacy weaknesses from the TAIP. The key insight of Taifu is that the applets which actually program the trigger-action rules can be used as test cases to explore the behavior of the TAIP. We evaluate the effectiveness of our approach by applying it on the TAIPs that are built around the IFTTT platform. To our great surprise, we find that privacy violations are prevalent among them. Using the automatically generated 407 applets, each from a different TAIP, Taifu detects 194 cases with access policy breaches, 218 access control missing, 90 access revocation missing, 15 unintended flows, and 73 over-privilege access.

Various third-party single sign-on (SSO) services (e.g., Facebook Login and Twitter Login) are widely deployed by web applications to facilitate their authentication and authorization processes. Nevertheless, integrating these services in a secure manner remains challenging, such that security issues are continually reported in recent years. In this work, we develop MoScan, a model-based scanner that can be used by software testers and security analysts for detecting and reporting security vulnerabilities in SSO implementations. MoScan takes as input a state machine built based on an SSO standard and our empirical study to represent participants' states and transitions during the login process. In the testing process, it analyzes network traces captured during the execution of SSO services, and increments the state machine which is then used to generate payloads to test the protocol participants. We evaluate MoScan with 23 real-world websites which integrate the Facebook SSO service to test its capability of identifying security vulnerabilities. To show the adaptability of MoScan's state machine, we also test it on Twitter and LinkedIn’s SSO services, and Github's authentication plugin in Jenkins. It detects three known weaknesses and one new logic fault from them, showing a new perspective in testing stateful protocol implementations like SSO services. Our demonstration and the source code of MoScan are available at https://github.com/baigd/moscan.

Deep Learning (DL) offers a data-driven programming paradigm in which Deep Neural Networks (DNNs) can be constructed through a set of training data.It has been widely adopted in many real-world applications.However, many studies have shown that DL systems suffer from adversarial attacks, especially when they are applied to security-and safetycritical domains.Given that formal verification has proved a great success in many areas such as software engineering, using it to achieve a high-level security assurance in DL systems is considered promising.In this paper, we design and implement DeepAuto which makes the significant bridge between automata and DNNs.With the aid of DeepAuto, we demonstrate how DNNs can be modeled as automata and be verified formally in the widely used model checker UPPAAL.The potential usefulness of DeepAuto shows the connection between DNNs and automata and provides a solution for the construction of more trustworthy DL systems.

The ongoing COVID-19 pandemic has impact almost every aspect of human lives profoundly.This paper investigates the impact of COVID-19 on the activity and contribution of open source software (OSS) developers.Specifically, we make great efforts to harvest the information of all the developers (over 25 million) on GitHub and their contribution activities.With such a large-scale dataset, we perform analysis from four perspectives, including the overall ecosystem level, country level, organization level and developer level, to characterize the impact of COVID-19 on the OSS community.We have revealed a number of interesting observations and trends, which are crucial to understanding the OSS contributors and supporting the collaboration to combat global crisis like COVID-19.

The website fingerprinting (or inter-domain WSF), enhanced by various machine learning techniques, has shown its power to identify websites a user has visited. To our best knowledge, a finer-grained problem of web page fingerprinting (or intra-domain WPF) has not been systematically studied by our research community. The WPF attackers, such as government agencies enforcing Internet censorship, are keen to identify the particular web pages (e.g., a political dissident’s social media page) visited by the target user.

A key feature of the booming smart home is the integration of a wide assortment of technologies, including various standards, proprietary communication protocols and heterogeneous platforms. Due to customization, unsatisfied assumptions and incompatibility in the integration, critical security vulnerabilities are likely to be introduced by the integration. Hence, this work addresses the security problems in smart home systems from anintegrationperspective, as a complement to numerous studies that focus on the analysis of individual techniques. We propose HomeScan, an approach that examines the security of the implementations of smart home systems. It extracts the abstract specification of application-layer protocols and internal behaviors of entities, so that it is able to conduct an end-to-end security analysis against various attack models. Applying HomeScanon three extensively-used smart home systems, we have found twelve non-trivial security issues, which may lead to unauthorized remote control and credential leakage.

Cryptocurrency has seen an explosive growth in recent years, thanks to the evolvement of blockchain technology and its economic ecosystem. Besides Bitcoin, thousands of cryptocur-rencies have been distributed on blockchains, while hundreds of cryptocurrency exchanges are emerging to facilitate the trading of digital assets. At the same time, it also attracts the attentions of attackers. Fake deposit, as one of the most representative attacks (vulnerabilities) related to exchanges and tokens, has been frequently observed in the blockchain ecosystem, causing large financial losses. However, besides a few security reports, our community lacks the understanding of this vulnerability, for example its scale and the impacts. In this paper, we take the first step to demystify the fake deposit vulnerability. Based on the essential patterns we have summarized, we implement DEPOSafe, an automated tool to detect and verify (exploit) the fake deposit vulnerability in ERC-20 smart contracts. DEPOSafe incorporates several key techniques including symbolic execution based static analysis and behavior modeling based dynamic verification. By applying DEPOSafe to 176,000 ERC-20 smart contracts, we have identified over 7,000 vulnerable contracts that may suffer from two types of attacks. Our findings demonstrate the urgency to identify and prevent the fake deposit vulnerability.

This paper presents Foggy, an anonymous communication system focusing on providing users with anonymous web browsing. Foggy provides a microservice-based proxy for web browsing and other low-latency network activities without exposing users' metadata and browsed content to adversaries. It is designed with decentralized information management, web caching, and configurable service selection. Although Foggy seems to be more centralized compared with Tor, it gains an advantage in manageability while retaining anonymity. Foggy can be deployed by several agencies to become more decentralized. We prototype Foggy and test its performance. Our experiments show Foggy's low latency and deployability, demonstrating its potential to be a commercial solution for real-world deployment.

Industrial control systems (ICS) of critical infrastructure are increasingly connected to the Internet for remote site management at scale. However, cyber attacks against ICS - especially at the communication channels between human-machine interface (HMIs) and programmable logic controllers (PLCs) - are increasing at a rate which outstrips the rate of mitigation. In this paper, we introduce a vendor-agnostic analytics framework which allows security researchers to analyse attacks against ICS systems, even if the researchers have zero control automation domain knowledge or are faced with a myriad of heterogenous ICS systems. Unlike existing works that require expertise in domain knowledge and specialised tool usage, our analytics framework does not require prior knowledge about ICS communication protocols, PLCs, and expertise of any network penetration testing tool. Using `digital twin' scenarios comprising industry-representative HMIs, PLCs and firewalls in our test lab, our framework's steps were demonstrated to successfully implement a stealthy deception attack based on false data injection attacks (FDIA). Furthermore, our framework also demonstrated the relative ease of attack dataset collection, and the ability to leverage well-known penetration testing tools. We also introduce the concept of `heuristic inference attacks', a new family of attack types on ICS which is agnostic to PLC and HMI brands/models commonly deployed in ICS. Our experiments were also validated on a separate ICS dataset collected from a cyber-physical scenario of water utilities. Finally, we utilized time complexity theory to estimate the difficulty for the attacker to conduct the proposed packet analyses, and recommended countermeasures based on our findings.

Mobile push notifications have been widely used in mobile platforms to deliver all sorts of information to app users. Although it offers great convenience for both app developers and mobile users, this feature was frequently reported to serve malicious and aggressive purposes, such as delivering annoying push notification advertisement. However, to the best of our knowledge, this problem has not been studied by our research community so far. To fill the void, this paper presents the first study to detect aggressive push notifications and further characterize them in the global mobile app ecosystem on a large scale. To this end, we first provide a taxonomy of mobile push notifications and identify the aggressive ones using a crowdsourcing-based method. Then we propose sc DaPanda, a novel hybrid approach, aiming at automatically detecting aggressive push notifications in Android apps. sc DaPanda leverages a guided testing approach to systematically trigger and record push notifications. By instrumenting the Android framework, sc DaPanda further collects all notification-relevant runtime information to flag the aggressive ones. Our experimental results show that sc DaPanda is capable of detecting different types of aggressive push notifications effectively in an automated way. By applying sc DaPanda to 20,000 Android apps from different app markets, it yields over 1,000 aggressive notifications, which have been further confirmed as true positives. Our in-depth analysis further reveals that aggressive notifications are prevalent across different markets and could be manifested in all the phases in the lifecycle of push notifications. It is hence urgent for our community to take actions to detect and mitigate apps involving aggressive push notifications.

A key feature of the booming smart home is the integration of a wide assortment of technologies, including various standards, proprietary communication protocols and heterogeneous platforms. Due to customization, unsatisfied assumptions and incompatibility in the integration, critical security vulnerabilities are likely to be introduced by the integration. Hence, this work addresses the security problems in smart home systems from an integration perspective, as a complement to numerous studies that focus on the analysis of individual techniques. We propose HOMESCAN, an approach that examines the security of the implementations of smart home systems. It extracts the abstract specification of application-layer protocols and internal behaviors of participants, so that it is able to conduct an end-to-end security analysis against various attack models. Applying HOMESCAN on three extensively-used smart home systems, we have found twelve non-trivial security vulnerabilities, which may lead to unauthorized remote control and credential leakage.

Blockchain technology relies on consensus algorithms to resolve conflicts in Byzantine environments. New blockchain algorithms are rapidly designed and implemented without a properly conducted formal analysis and verification. In this paper, we conducted a study on Tendermint which is a proof-of-stake consensus algorithm. We verified that the consensus protocol is deadlock-free and is able to reach consensus when at least 2/3 of the network is in agreement. We also proved that a minority set of nodes that compose more than 1/3 of the network is enough to censor the majority of the network and prevent the network from reaching consensus and conclude that the algorithm has some shortcomings on availability.

Dynamic slicing is a common way of identifying the root cause when a program fault is revealed. With the dynamic slicing technique, the programmers can follow data and control flow along the program execution trace to the root cause. However, the technique usually fails to work on omission bugs, i.e., the faults which are caused by missing executing some code. In many cases, dynamic slicing over-skips the root cause when an omission bug happens, leading the debugging process to a dead end. In this work, we conduct an empirical study on the omission bugs in the Defects4J bug repository. Our study shows that (1) omission bugs are prevalent (46.4%) among all the studied bugs; (2) there are repeating patterns on causes and fixes of the omission bugs; (3) the patterns of fixing omission bugs serve as a strong hint to break the slicing dead end. Based on our findings, we train a neural network model on the omission bugs in Defects4J repository to recommend where to approach when slicing can no long work. We conduct an experiment by applying our approach on 3193 mutated omission bugs which slicing fails to locate. The results show that our approach outperforms random benchmark on breaking the dead end and localizing the mutated omission bugs (63.8% over 2.8%).

A precise representation for attacks can benefit the detection of malware in both accuracy and efficiency. However, it is still far from expectation to describe attacks precisely on the Android platform. In addition, new features on Android, such as communication mechanisms, introduce new challenges and difficulties for attack detection. In this paper, we propose abstract attack models to precisely capture the semantics of various Android attacks, which include the corresponding targets, involved behaviors as well as their execution dependency. Meanwhile, we construct a novel graph-based model called the inter-component communication graph (ICCG) to describe the internal control flows and inter-component communications of applications. The models take into account more communication channel with a maximized preservation of their program logics. With the guidance of the attack models, we propose a static searching approach to detect attacks hidden in ICCG. To reduce false positive rate, we introduce an additional dynamic confirmation step to check whether the detected attacks are false alarms. Experiments show that DroidEcho can detect attacks in both benchmark and real-world applications effectively and efficiently with a precision of 89.5%.

As feature-rich Android applications (apps for short) are increasingly popularized in security-sensitive scenarios, methods to verify their security properties are highly desirable. Existing approaches on verifying Android apps often have limited effectiveness. For instance, static analysis often suffers from a high false-positive rate, whereas approaches based on dynamic testing are limited in coverage. In this work, we propose an alternative approach, which is to apply the software model checking technique to verify Android apps. We have built a general framework named DroidPF upon Java PathFinder (JPF), towards model checking Android apps. In the framework, we craft an executable mock-up Android OS which enables JPF to dynamically explore the concrete state spaces of the tested apps; we construct programs to generate user interaction and environmental input so as to drive the dynamic execution of the apps; and we introduce Android specific reduction techniques to help alleviate the state space explosion. DroidPF focuses on common security vulnerabilities in Android apps including sensitive data leakage involving a non-trivial flow- and context-sensitive taint-style analysis. DroidPF has been evaluated with 131 apps, which include real-world apps, third-party libraries, malware samples and benchmarks for evaluating app analysis techniques like ours. DroidPF precisely identifies nearly all of the previously known security issues and nine previously unreported vulnerabilities/bugs.

For industrial control systems, ensuring the software integrity of their devices is a key security requirement. A pure software-based attestation solution is highly desirable for protecting legacy field devices that lack hardware root of trust (e.g., Trusted Platform Module). However, for the large population of field devices with ARM processors, existing software-based attestation schemes either incur long attestation time or are insecure. In this paper, we design a novel memory stride technique that significantly reduces the attestation time while remaining secure against known attacks and their advanced variants on ARM platform. We analyze the scheme's security and performance based on the formal framework proposed by Armknecht et al. [7] (with a necessary change to ensure its applicability in practical settings). We also implement memory stride on two models of real-world power grid devices that are widely deployed today, and demonstrate its superior performance.

The heavy use of event-callback mechanism in frameworks like Android causes challenges for static analysis. Modelling of callback mechanisms for Android applications (app for short) is becoming a major method to address such challenges. In this work, we aim to construct a generic callback-related model that supports path-sensitive analysis. We consider three unresolved challenges in the existing modelling approaches: 1) building connections between different components; 2) identifying path-sensitive conditions; 3) handling the system-driven callbacks and fine-grained lifecycle callbacks. We propose algorithms for constructing a generic path-sensitive callback model and present a prototype model constructor, AndroChecker, to validate our approach. We evaluate 20 real-world apps using AndroChecker. The evaluation result shows that our method and tool have a strong capability in modelling path conditions and inter-component invocations.

Concurrent Programs are hard to analyze or debug due to the complex program logic and unpredictable execution environment. In practice, ordinary programmers often adopt existing well-designed concurrency related API (e.g., those in java.util.concurrent) so as to avoid dealing with these issues. These API can however often be used incorrectly, which results in hardto-debug concurrent bugs. In this work, we propose an approach for enforcing the correct usage of concurrency-related Java API. Our idea is to annotate concurrency-related Java classes with annotations related to misuse of these API and develop lightweight type checker to detect concurrent API misuse based on the annotations. To automate this process, we need to solve two problems: (1) how do we obtain annotations of the relevant API; and (2) how do we systematically detect concurrent API misuse based on the annotations? We solve the first problem by extracting annotations from the API documentation using natural language processing techniques. We solve the second problem by implementing our type checkers in the Checker Framework to detect concurrent API misuse. We apply our approach to extract annotations for all classes in the Java standard library and use them to detect concurrent API misuse in open source projects on GitHub. We confirm that concurrent API misuse is common and often results in bugs or inefficiency.

Abstract The peer-assisted CDN is a new content distribution paradigm supported by CDNs (e.g., Akamai), which enables clients to cache and distribute web content on behalf of a website. Peer-assisted CDNs bring significant bandwidth savings to website operators and reduce network latency for users. In this work, we show that the current designs of peer-assisted CDNs expose clients to privacy-invasive attacks, enabling one client to infer the set of browsed resources of another client. To alleviate this, we propose an anonymous peer-assisted CDN (APAC), which employs content delivery while providing initiator anonymity (i.e., hiding who sends the resource request) and responder anonymity (i.e., hiding who responds to the request) for peers. APAC can be a web service, compatible with current browsers and requiring no client-side changes. Our anonymity analysis shows that our APAC design can preserve a higher level of anonymity than state-of-the-art peer-assisted CDNs. In addition, our evaluation demonstrates that APAC can achieve desired performance gains.

Recent studies on the Android permission system have found that there exists a permission gap between the requested permissions and permissions actually used in an Android app. However, current approaches face some challenges when detecting such permission gaps in Android apps due to the limitation of static analysis techniques. This paper proposes a novel approach to detect permission gaps in Android apps and determine the precise set of permissions that an app needs to run correctly. Our approach includes a static analysis technique to extract permission usage information from API invocations, and a dynamic testing technique to test and monitor the runtime permission usage behaviors of apps. By combining static analysis and dynamic testing, our approach can detect significantly more permission usage information compared to static analysis, indicating that our approach could improve the detection accuracy and reduce the false positives in permission gap detection. We have implemented a prototype to study more than 1,000 popular apps from Google Play. The results show that our approach could detect on average 30% more permissions that are used in apps, while more than 8% of the overprivileged apps detected by previous approaches are false positives.

Security of authentication protocols heavily relies on the confidentiality of credentials (or authenticators) like passwords and session IDs. However, unlike browser-based web applications for which highly evolved browsers manage the authenticators, Android apps have to construct their own management. We find that most apps simply locate their authenticators into the persistent storage and entrust underlying Android OS for mediation. Consequently, these authenticators can be leaked through compromised backup channels. In this work, we conduct the first systematic investigation on this previously overlooked attack vector. We find that nearly all backup apps on Google Play inadvertently expose backup data to any app with internet and SD card permissions. With this exposure, the malicious apps can steal other apps' authenticators and obtain complete control over the authenticated sessions. We show that this can be stealthily and efficiently done by building a proof-of-concept app named AuthSniffer. We find that 80 (68.4%) out of the 117 tested top-ranked apps which have implemented authentication schemes are subject to this threat. Our study should raise the awareness of app developers and protocol analysts about this attack vector.

As the boom of social networking, Single Sign-On (SSO) services developed by major commercial service providers like Facebook, Google and Twitter, have been widely used by web-based service providers as an alternative authentication scheme. Despite rich research has focused on browser-based web applications, little has been conducted on the implementation of SSO on mobile platforms. However, we reveal that due to the fundamental difference of isolation mechanism in mobile OS and applications from the origin-based isolation in browsers, the SSO encounters a novel attack surface and adversarial models. We perform the first formal analysis on the implementation of the most widely used SSO service -- Facebook Login. Our study takes as input the available implementation and dynamic execution traces of Facebook SDK for Android, from which we abstract the implementation-level protocol. The protocol is then modeled in typed Pi-calculus, and automatically checked against the mobile platform specific attack models in a protocol verifier Proverif. Our study has successfully identified a major vulnerability, which allows an attacker to steal authentication credentials from victims and log into their Facebook accounts.

Mobile OSes and applications form a large, complex and vulnerability-prone software stack. In such an environment, security techniques to strongly protect sensitive data in mobile devices are important and challenging. To address such challenges, we introduce the concept of the trusted data vault, a small trusted engine that securely manages the storage and usage of sensitive data in an untrusted mobile device. In this paper, we design and build Droid Vault - the first realization of a trusted data vault on the Android platform. Droid Vault establishes a secure channel between data owners and data users while allowing data owners to enforce strong control over the sensitive data with a minimal trusted computing base (TCB). We prototype Droid Vault via the novel use of hardware security features of ARM processors, i.e., Trust Zone. Our evaluation demonstrates its functionality for processing sensitive data and its practicality for adoption in the real world.

ABSTRACT The Web of Things (WoT), inherited from the Internet of Things (IoT), encapsulates functionalities into publishable services on the Web to enable the IoT a seamless integration with the Web. The openness of the Web, in turn, directly exposes WoT to existing attacks from the Web. In addition, WoT possesses characteristics of high security and privacy concerns, mobility, and limited capabilities, which require specific and additional security and privacy protection beyond existing mechanisms. More importantly, WoT is inherently connected to its context, so context information must be taken into account in its security and privacy measures. To address these challenges, we propose a context‐aware usage control model (ConUCON), which leverages the context information to enhance data, resource, and service protection for WoT. On the basis of ConUCON, we also design and implement a context‐aware usage control framework on the middleware layer in our ongoing SmartHome project, to provide security and privacy protection. ConUCON is designed specifically to express the context‐aware usage policy specification, such that security and privacy requirements can be easily specified and enforced with the proposed model and framework. Finally, we apply ConUCON to a remote appliance management prototype, as a case study, to demonstrates its feasibility in a real environment. Copyright © 2012 John Wiley & Sons, Ltd.

In the Android system, applications (apps) execute on the same platform that manages all system resources, where resource accesses are regulated through a permission-based mechanism. As a result, malicious apps get chances to abuse resources that are available on the Android platform. In this paper, we propose resource virtualization as a security mechanism to confine resource-abusing Android apps. The physical resources on a mobile device are virtualized to a different virtual view for selected Android apps. Resource virtualization simulates a partial but consistent virtual view of the Android resources. Therefore, it can not only confine the resource-abusing apps effectively, but also ensure the usability of them. We implement a system prototype, RVDroid, and evaluate it with real-world apps of various types. Our results demonstrate its effectiveness on malicious Android apps and its compatibility and usability on benign ones.