Developing Medical AI relies on large datasets and easily suffers from data scarcity. Generative data augmentation (GDA) using AI generative models offers a solution to synthesize realistic medical images. However, the bias in GDA is often underestimated in medical domains, with concerns about the risk of introducing detrimental features generated by AI and harming downstream tasks. This paper identifies the frequency misalignment between real and synthesized images as one of the key factors underlying unreliable GDA and proposes the Frequency Recalibration (FreRec) method to reduce the frequency distributional discrepancy and thus improve GDA. FreRec involves (1) Statistical High-frequency Replacement (SHR) to roughly align high-frequency components and (2) Reconstructive High-frequency Mapping (RHM) to enhance image quality and reconstruct high-frequency details. Extensive experiments were conducted in various medical datasets, including brain MRIs, chest X-rays, and fundus images. The results show that FreRec significantly improves downstream medical image classification performance compared to uncalibrated AI-synthesized samples. FreRec is a standalone post-processing step that is compatible with any generative model and can integrate seamlessly with common medical GDA pipelines.

Large Vision-Language Models (LVLMs) rely on attention-based retrieval of safety instructions to maintain alignment during generation.Existing attacks typically optimize image perturbations to maximize harmful output likelihood, but suffer from slow convergence due to gradient conflict between adversarial objectives and the model's safety-retrieval mechanism.We propose Attention-Guided Visual Jailbreaking, which circumvents rather than overpowers safety alignment by directly manipulating attention patterns.Our method introduces two simple auxiliary objectives: (1) suppressing attention to alignment-relevant prefix tokens and (2) anchoring generation on adversarial image features.This simple yet effective push-pull formulation reduces gradient conflict by 45% and achieves 94.4% attack success rate on Qwen-VL (vs.68.8% baseline) with 40% fewer iterations.At tighter perturbation budgets (ϵ=8/255), we maintain 59.0% ASR compared to 45.7% for standard methods.Mechanistic analysis reveals a failure mode we term safety blindness: successful attacks suppress system-prompt attention by 80%, causing models to generate harmful content not by overriding safety rules, but by failing to retrieve them.

This research introduces a novel intrusion detection method that examines network traffic from Internet of Things (IoT) devices. Due to their constrained computational resources, IoT devices are inherently more vulnerable to cyber-attacks compared to traditional computing systems. Botnets, which often orchestrate distributed denial-of-service (DDoS) attacks by leveraging numerous IoT devices, pose a significant security threat. Consequently, developing effective strategies to identify and mitigate botnet impacts within IoT environments is crucial. Our approach employs an IP and port-based classification model capable of recognizing previously unseen intrusion types post-deployment. By monitoring changes in device behavior, the system effectively distinguishes between normal and anomalous activities.We demonstrate the efficacy of our method by targeting two prominent IoT botnets: Mirai and Bashlite. Additionally, we evaluate the benefits of integrating bootstrapping with averaging techniques during the preprocessing phase, finding that this combination significantly enhances the model’s generalizability. The MOAT framework exhibits outstanding performance in both localized and federated intrusion detection scenarios, achieving an average accuracy of 96.25% across nodes, even when tested against attack types present in the training data.

Learned indexes leverage machine learning models to approximate data distributions and predict key positions, offering better performance than traditional index structures such as B+Trees. As data in real-world applications evolve rapidly, the timely and efficient updating of learned indexes has become an increasingly important research problem, attracting growing attention in recent studies. However, under update-intensive workloads with frequent insertions and deletions, existing learned indexes cannot update the model in a timely manner. Moreover, they ignore the impact of deletions on model accuracy. These limitations lead to degraded prediction accuracy and increased query latency, undermining the core advantage of learned indexes. To fill in this gap, in this paper, we design a dynamic learned index (denoted as DynaMind) that is able to timely update the model with the frequent change of data. Specifically, we propose a novel score function to determine the appropriate timing at which a learned index should initiate an update by measuring the influence of updated data on the model accuracy. To enable efficient model updates, we devise a timely learned index update algorithm that implements both lightweight incremental learning for insertions and machine unlearning for deletions together, ensuring the model continuously evolves without full retraining. Extensive experiments on real-world and synthetic datasets show that DynaMind achieves competitive throughput compared to state-of-the-art works while improving the prediction accuracy. The proportion of keys with zero prediction error increases by more than 10% after updates.

Unsupervised time series representation learning, driven by recent advances in contrastive learning-based methods, has become a critical component for downstream tasks like forecasting and classification. However, time series data exhibit complex temporal dependencies and spectral patterns, posing challenges for existing approaches to adapt robustly. Moreover, existing contrastive learning-based approaches overlook frequency-domain information and struggle with selecting effective negative samples, further hindering model performance. To address these issues, we propose S a URL-TS, a novel self-adaptive framework for unsupervised time series representation learning. First, it dynamically learns dataset-specific augmentations to generate high-quality positive samples. Second, an adaptive self-supervised learning module with a multi-domain encoder captures both temporal and spectral patterns without relying on negative samples. Third, a representation-wise attention mechanism assigns dynamic weights to representations across domains. To the best of our knowledge, S a URL-TS is the first self-supervised learning framework to jointly model temporal and spectral patterns across both augmentation and learning stages. Extensive experiments confirm the superior performance of S a URL-TS over state-of-the-art models. Notably, its adaptive data augmentation module is plug-and-play and can be integrated into other contrastive learning frameworks, and its learning stage is capable of adapting to a wide range of time series patterns. Our codebase is available at https://github.com/YusenL/SAURL-TS .

Data augmentation in reinforcement learning (RL) aims to generate diverse and extensive datasets to enhance the learning process. Most existing studies on RL augmentation employ sample-based approaches that modify existing samples. However, many of these methods directly adopt augmentation strategies from other domains, which may present limitations when applied to RL. For instance, approaches derived from computer vision techniques are often not well-suited for general RL applications that do not involve visual inputs. Moreover, such sample-based methods frequently overlook the broader characteristics of the training environment, rendering the augmented data potentially less effective in complex scenarios, such as intelligent transportation systems. In addition, these methods can introduce significant risks to critical and sensitive data. For example, introducing noise to samples as a method of augmentation can precipitate adversarial attacks. Thus, a reliable and stable method of augmentation is necessary. To address these concerns, we propose a novel large language model (LLM)-based augmentation strategy in RL with machine unlearning and fine-tuning. This method utilizes LLMs for data augmentation, ensuring the reliability of the augmented data by tailoring it to specific environmental contexts. Additionally, it enhances the quality of augmentation by mitigating the impact on critical samples through the proposed novel machine unlearning method, while simultaneously fine-tuning the model to improve overall performance. Our experimental results indicate that this innovative approach significantly surpasses traditional augmentation methods in learning performance. By mitigating the impact on critical samples, our strategy not only generates more reliable augmented data but also enhances the effectiveness of RL models.

Machine unlearning is a technique that removes specific data influences from trained models without the need for extensive retraining. However, it faces several key challenges, including the lack of explanation, privacy concerns during the unlearning process, and the high demand for time and computational resources. This paper presents a novel unlearning approach to tackle above challenges by employing Layer-wise Relevance Analysis and Neuronal Path Perturbation. Our method balances machine unlearning performance and model utility by identifying and perturbing highly relevant neurons, thus achieving effective unlearning. Using unseen data that has not been presented in the original training set, our method achieves zero-shot unlearning, which allows for the removal of specific class knowledge without accessing the original training data during the unlearning process. This approach ensures robust privacy protection. Experimental results demonstrate that our approach effectively removes targeted data from the target unlearning model while maintaining the model's utility, offering a practical solution for privacy-preserving machine learning. Our code is available athttps://github.com/ChangWenhan/LRA-NPP-Unlearning

Artificial Intelligence (AI) technology has been widely used in malware detection and has significantly improved defense against cyberattacks. Existing deep learning-based methods rely on training with large-scale data and only on predefined categories, making them inadequate for rapidly responding to novel malware attacks. Malware classification based on few-shot learning has made some progress in identifying unknown malware using limited data. However, existing methods struggle to achieve high performance because they typically focus on a single malicious feature, such as a single malware image or an API call sequence, thereby ignoring the multi-dimensional nature of malware. To deal with these challenges, we propose a multi-view few-shot learning method for malware classification. We propose a multi-view malicious feature engineering scheme, which combines domain knowledge and expert experience to analyze the malware from various perspectives. Furthermore, we propose a support-query prototype generation method based on multi-view malicious features to generate higher-quality malware prototypes, which enhances the representation of novel malware family distributions. Extensive experiments show that the proposed method outperforms existing state-of-the-art approaches. With only two samples per family, the accuracy still exceeds 90%. Our method demonstrates superior cross-dataset recognition capabilities, thereby fully illustrating its robustness and generalizability across different data distributions.

Large-scale text-to-image (T2I) diffusion models have revolutionized image generation, enabling the synthesis of highly detailed visuals from textual descriptions. However, these models may inadvertently generate inappropriate content, such as copyrighted works or offensive images. While existing methods attempt to eliminate specific unwanted concepts, they often fail to ensure robust removal-allowing the concept to reappear in subtle forms. For instance, a model may successfully avoid generating images in Van Gogh's style when explicitly prompted with “Van Gogh”, yet still reproduce his signature artwork when given the prompt “Starry Night”. In this paper, we propose SAFER, a novel and efficient approach for thoroughly removing target concepts from diffusion models. At a high level, SAFER is inspired by the observed low-dimensional structure of the text embedding space. The method first identifies a concept-specific subspace$\mathcal {S}_{c}$associated with the target concept$c$. It then projects the prompt embeddings onto the complementary subspace of$\mathcal {S}_{c}$, effectively erasing the concept from the generated images. Since concepts can be abstract and difficult to fully capture using natural language alone, we employ textual inversion to learn an optimized embedding of the target concept from a reference image. This enables more precise subspace estimation and enhances removal performance. Furthermore, we introduce a subspace expansion strategy to ensure comprehensive and robust concept erasure. Extensive experiments demonstrate that SAFER consistently and effectively erases unwanted concepts from diffusion models while preserving generation quality.

Auditing differential privacy (DP) guarantees often relies on querying trained models with specially crafted queries, such as canaries, examples differing between two neighboring datasets. However, in this work, we revisit this common approach and identify a fundamental limitation: canary-based queries may not capture the strongest privacy leakage, as the most informative queries can shift during the training process. This mismatch can result in loose lower bounds on the privacy parameter$\varepsilon$, underestimating potential risks from query-based adversaries. To address this issue, we propose two methods. First, we introduce a consistent and optimizable surrogate privacy loss function that better aligns with the true privacy loss, called Privacy-loss Maximization Method (PMM), enabling systematic discovery of stronger queries through optimization. Second, we analyze how the optimal queries evolve with model training and propose a gradient-aligned query generation algorithm, called Gradient-Guided Querying (GGQ), that rapidly identifies high-risk queries by aligning their gradients with the distribution of model parameters. Empirical evaluations across multiple tasks demonstrate that our methods consistently produce stronger privacy audit results, offering a more accurate assessment of the privacy risks associated with training algorithms.

Due to the growing emphasis on privacy and data governance in machine learning, federated unlearning, an emerging concept in the domain of federated learning, stems from the growing need to address the dynamic nature of data and the evolving requirements related to privacy, compliance, and data management. However, there are some security risks during the unlearning process, including the potential for adversarial manipulation of model integrity, privacy breaches, and performance degradation in a federated learning framework. Although existing research has proposed various defenses to mitigate these risks, significant vulnerabilities remain that can be exploited to undermine the integrity and effectiveness of the unlearning process. Current attack methods are limited by their detectability during training, lack of persistence, and reliance on test-time triggers, which reduces their overall effectiveness. In this paper, we introduce camouflaged poisoning attacks, a novel attack paradigm relevant to federated unlearning. In this approach, some adversary clients initially infuse a small number of meticulously designed points into the dataset, ensuring that the model's predictions are barely influenced. The adversary then makes a request to the exclusion of some of these malicious clients. At this juncture, the attack is activated, leading to a detrimental impact on the model's predictions. The outcomes reveal a substantial potential for these strategies to compromise the effectiveness of models in unlearning scenarios. The essence of this attack involves the creation of deceptive clients that conceal the influence of a contaminated dataset during the federated unlearning process.

To ensure data robustness, users typically create replicas of critical data and store them on multiple servers. However, due to server failures or malicious attacks, these replicas may face the risk of loss or tampering. Therefore, periodic auditing of data replicas is necessary to ensure their integrity. Existing solutions perform periodic auditing with generating metadata for each replica individually. However, generating auditing metadata for every replica incurs significant computational overhead, and storing metadata for multiple replicas also imposes a considerable storage burden. To reduce this overhead and optimize the efficiency of auditing towards multiple replicas, we propose collusion-resistant multi-replica data auditing with optimized metadata generation (CRMRDA). First, CRMRDA enhances replica generation efficiency by employing symmetric encryption and random masking. An indistinguishable challenge strategy is introduced, making the number of generated auditing metadata copies independent of the number of replicas. Second, we construct a security model for multi-replica data possession that allows server collusion and provide a formal security proof for CRMRDA. In addition, CRMRDA supports practical features such as dynamic operations and batch auditing. Finally, the efficiency of CRMRDA is evaluated in an experimental environment consisting of four servers, one third-party auditor, and one data owner. Compared to the latest multi-replica data auditing schemes, CRMRDA demonstrates lower computational overhead in replica generation, metadata generation, and periodic auditing.

The increasing demand for autonomous and open peer-to-peer (P2P) data sharing has driven the widespread adoption of decentralized file systems, such as the InterPlanetary File System (IPFS). However, decentralized data sharing inherently requires distributed access control mechanisms due to the absence of centralized authorities. Although blockchain-based access control has become a primary solution, the public nature of blockchain can unintentionally reveal user attributes, posing significant privacy risks. To address the leakage of attribute sets in blockchain, we propose zk-Guard, a decentralized access control framework integrating blockchain and zero-knowledge Succinct Non-interactive Arguments of Knowledge (zk-SNARKs) tailored for IPFS. To further improve the efficiency of zero-knowledge policy checking and reduce the delay of policy updating, we employ a universal constraint circuit and encode policies into sparse configuration matrices, achieving fine-grained, rapid policy updates without regenerating proving keys while guaranteeing constant-time verification regardless of policy complexity. Additionally, to prevent repeated permission checks for large f iles and improve system responsiveness, zk-Guard integrates Merkle Tree Proof (MTP) mechanisms to securely link sub-data blocks to their root block. Comprehensive theoretical complexity analysis and extensive experiments demonstrate that zk-Guard achieves substantial performance improvements over existing schemes, with constant-time proof verification under 2.5 ms enabling efficient data retrieval, and policy deployment and updates completed within 0.2 seconds even for 1,000 attributes. The source code is available at https://github.com/ningboliucug/zk-Guard.

As deep image forgery powered by AI generative models, such as GANs, continues to challenge today's digital world, detecting AI-generated forgeries has become a vital security topic. Generalizability and robustness are two critical concerns of a forgery detector, determining its reliability when facing unknown GANs and noisy samples in an open world. Although many studies focus on improving these two properties, the root causes of these problems have not been fully explored, and it is unclear if there is a connection between them. Moreover, despite recent achievements in addressing these issues from image forensic or anti-forensic aspects, a universal method that can contribute to both sides simultaneously remains practically significant yet unavailable. In this paper, we provide a fundamental explanation of these problems from a frequency perspective. Our analysis reveals that the frequency bias of a DNN forgery detector is a possible cause of generalization and robustness issues. Based on this finding, we propose a two-step frequency alignment method to remove the frequency discrepancy between real and fake images, offering double-sided benefits: it can serve as a strong black-box attack against forgery detectors in the anti-forensic context or, conversely, as a universal defense to improve detector reliability in the forensic context. We also develop corresponding attack and defense implementations and demonstrate their effectiveness, as well as the effect of the frequency alignment method, in various experimental settings involving twelve detectors, eight forgery models, and five metrics

Machine unlearning is an emerging technology that removes a subset of the training data from a trained model without significantly affecting the model performance on the remaining data. This topic is becoming increasingly important in protecting user privacy and eliminating harmful or outdated data. The key challenge lies in effectively and efficiently un learning specific information without compromising the model's utility on the retained data. For pre-trained models, fine-tuning is an important way to achieve the unlearning target. Previous work typically fine-tuned the entire model's parameters, which incurred significant computational costs. In addition, the fine tuning process may cause shifts in the intermediate layer features, affecting the model's overall utility. In this work, we propose a novel and efficient machine unlearning method for pre-trained models. We term the method Residual Feature Alignment Un learning. Specifically, we leverage LoRA (Low-Rank Adaptation) to decompose the model's intermediate features into pre-trained features and residual features. By adjusting the residual features, we align the unlearned model with the pre-trained model at the intermediate feature level to achieve both unlearning and remaining targets. The method aims to learn zero residuals on the retained set and shifted residuals on the unlearning set. Extensive experiments on numerous datasets validate the effectiveness of our approach.

With the rapid growth of mobile applications, large volumes of private data are continuously generated on mobile devices and increasingly outsourced to cloud storage services. Due to the limited computational and energy resources of mobile devices, enabling efficient and secure periodic auditing of cloud stored data remains a challenging task. Existing cloud data auditing schemes, such as Provable Data Possession and Proof of Retrievability, require computation-intensive operations that are impractical for resource-constrained mobile environments. To address this challenge, we propose MIDAS, a mobile-friendly data auditing scheme that supports secure outsourcing of auditing metadata with balanced resource utilization. MIDAS adopts a two-phase auditing metadata structure, in which computation intensive operations are offloaded to edge nodes, while mobile devices only perform lightweight processing. In addition, identity information of edge nodes is embedded into the auditing metadata to support traceability and accountability. To further improve scalability, we design a resource scheduling model for Multi MU and Multi-EN scenarios and propose the proportional load balanced scheduling algorithm to achieve efficient and load balanced metadata generation. The security of MIDAS is rigorously proven by reducing it to the discrete logarithm problem in the RSA quadratic residue subgroup within the random oracle model. Experimental results in a real-world environment demonstrate that MIDAS significantly reduces computational overhead on mobile devices, maintains acceptable communication cost, and achieves superior scalability compared with existing schemes.

The deployment of large language models (LLMs) like ChatGPT and Gemini has shown their powerful natural language generation capabilities. However, these models can inadvertently learn and retain sensitive information and harmful content during training, raising significant ethical and legal concerns. To address these issues, machine unlearning has been introduced as a potential solution. While existing unlearning methods take into account the specific characteristics of LLMs, they often suffer from high computational demands, limited applicability, or the risk of catastrophic forgetting. To address these limitations, we propose a lightweight behavioral unlearning framework based on Retrieval-Augmented Generation (RAG) technology. By modifying the external knowledge base of RAG, we simulate the effects of forgetting without directly interacting with the unlearned LLM. We approach the construction of unlearned knowledge as a constrained optimization problem, deriving two key components that underpin the effectiveness of RAG-based unlearning. This RAG-based approach is particularly effective for closed-source LLMs, where existing unlearning methods often fail. We evaluate our framework through extensive experiments on both open-source and closed-source models, including ChatGPT, Gemini, Llama-2-7b-chat, and PaLM 2. The results demonstrate that our approach meets five key unlearning criteria: effectiveness, universality, harmlessness, simplicity, and robustness. Meanwhile, this approach can extend to multimodal large language models and LLM-based agents.

Machine unlearning enables pre-trained models to eliminate the effects of partial training samples. Previous research has mainly focused on proposing efficient unlearning strategies. However, the verification of machine unlearning, or in other words, how to guarantee that a sample has been successfully unlearned, has been overlooked for a long time. Existing verification schemes typically rely on machine learning attack techniques, such as backdoor attacks or membership inference attacks. As these techniques are not formally designed for verification, they are easily bypassed when an untrustworthy model provider in MLaaS undergoes rapid fine-tuning to meet the verification conditions only, rather than executingrealunlearning. In this paper, we propose a formal verification scheme, IndirectVerify, to determine whether unlearning requests have been successfully executed. We design influential sample pairs: one referred to astrigger samplesand the other asreaction samples. Users send unlearning requests regarding trigger samples and use reaction samples to verify if the unlearning operation has been successfully carried out. We propose a perturbation-based scheme to generate those influential sample pairs. The objective is to perturb only a small fraction of training samples to trigger samples, leading to the misclassification of reaction samples. This indirect influence will be used for our verification purposes. In contrast to existing schemes that employ the same samples for all processes, our scheme, IndirectVerify, provides enhanced robustness, making it less susceptible to bypassing processes.

The rapid evolution of digital currency trading, fueled by the integration of blockchain technology, has led to both innovation and the emergence of smart Ponzi schemes. A smart Ponzi scheme is a fraudulent investment operation in smart contract that uses funds from new investors to pay returns to earlier investors. Traditional Ponzi scheme detection methods based on deep learning typically rely on fully supervised models, which require large amounts of labeled data. However, such data is often scarce, hindering effective model training. To address this challenge, we propose a novel contrastive learning framework, CASPER (Contrastive Approach for Smart Ponzi detectER with more negative samples), designed to enhance smart Ponzi scheme detection in blockchain transactions. By leveraging contrastive learning techniques, CASPER can learn more effective representations of smart contract source code using unlabeled datasets, significantly reducing both operational costs and system complexity. We evaluate CASPER on the XBlock dataset, where it outperforms the baseline by 2.3% in F1 score when trained with 100% labeled data. More impressively, with only 25% labeled data, CASPER achieves an F1 score nearly 20% higher than the baseline under identical experimental conditions. These results highlight CASPER's potential for effective and cost-efficient detection of smart Ponzi schemes, paving the way for scalable fraud detection solutions in the future.

Multi-LLM collaborative systems have attracted significant attention as a promising solution for complex tasks, enabling multiple large language models (LLMs) with different domains to work together toward a common goal. Different collaborative structures (e.g., Centralized, Horizontal, and Joint Interaction) and communication methods (e.g., direct, summary, and vote) give the system with enhanced flexibility and reasoning capability. However, these same mechanisms also introduce potential security and privacy risks, such as the generation of incorrect responses and the leakage of sensitive information. Based on the above unique characteristics of collaborative systems, we propose three attack methods (named Decision Poisoning Attack, Indirect Echoleak Attack and Information Collision Attack) that exploit the interactions between LLMs to achieve different objectives like system manipulation and privacy leakage. Extensive experiments demonstrate the effectiveness of the proposed attack on three structures and three communication methods, highlighting the security vulnerabilities and potential risks in Multi-LLM collaborative systems. We further discuss possible defense methods that can mitigate the attack performance. Our work show that (1) the key factor for a successful attack on collaborative systems is ensuring the malicious instruction persists and propagates throughout the inter-LLMs communication. (2) both the system architecture and the communication method can affect attack effectiveness, offering valuable insights for the design of more secure Multi LLM collaborative systems in the future.

The development of Large Language Models (LLMs) faces a significant challenge: the exhaustion of publicly available fresh data. This is because training an LLM requires a large demand for new data. Federated learning emerges as a promising solution, enabling collaborative model to contribute their private data to LLM global model. However, integrating federated learning with LLMs introduces new challenges, including the lack of transparency and the need for effective unlearning mechanisms. Transparency is essential to ensuring trust and fairness among participants, while accountability is crucial for deterring malicious behaviour and enabling corrective actions when necessary. To address these challenges, we propose a novel blockchain-based federated learning framework for LLMs that enhances transparency, accountability, and unlearning capabilities. Our framework leverages blockchain technology to create a tamper- proof record of each model's contributions and introduces an innovative unlearning function that seamlessly integrates with the federated learning mechanism. We investigate the impact of Low-Rank Adaptation (LoRA) hyperparameters on unlearning performance and integrate Hyperledger Fabric to ensure the security, transparency, and verifiability of the unlearning process. Through comprehensive experiments and analysis, we showcase the effectiveness of our proposed framework in achieving highly effective unlearning in LLMs trained using federated learning. Our findings highlight the feasibility of integrating blockchain technology into federated learning frameworks for LLM.

Source code vulnerabilities represent a critical threat to software security, potentially leading to severe consequences such as data breaches and system failures. Traditional static analysis tools, while widely used, suffer from high false positive rates and struggle to adapt to the increasing complexity of modern software. Deep learning-based approaches hold promise for automated vulnerability detection, but they face challenges including limited dataset quality, inadequate feature extraction, and lack of precise vulnerability localization capabilities. To address these limitations, we propose MaliVD, a novel vulnerability detection method in source code that leverages a multi-modal attention mechanism. MaliVD not only identifies vulnerability types but also pinpoints the specific lines of code where vulnerabilities are triggered. The model extracts sequential, tree-based, and graph-based features from source code and employs specialized neural networks to learn these diverse representations. By strategically focusing on Points of Interest within the code, MaliVD effectively prioritizes potentially vulnerable code regions, enhancing both detection accuracy and localization precision. Experimental results show that when compared with eight advanced vulnerability detection models across three large datasets, MaliVD demonstrates superior vulnerability detection and localization capabilities, maintaining highF1scores and localization precision. Particularly on the ReliVul dataset, theF1score is improved by 21.82%, and the Top-5 localization accuracy is 18% higher than other methods, with lower false positives across six mainstream vulnerability types, validating MaliVD’s practical application value in real-world environments.

In recent years, advancements in large language models have led to significant innovation and critical progress in AI. However, some of these innovations are raising privacy and security concerns. Machine unlearning has therefore emerged as a potential solution to mitigate such risks. Yet, while erasing data records from traditional models is relatively straightforward, making a large language model “forget” what it has learned is often very challenging. This is not just because they include so many parameters, it is also because the knowledge they possess is intricately entangled. Further, the privacy risk of unlearned data remains neglected in most unlearning solutions. To overcome these limitations, we took advantage of information retrieval and developed an efficient privacy-preserving unlearning mechanism. Our solution eliminates the impact of targeted information by removing high-risk semantic meanings from the model’s output. It also incorporates differentially-private randomization to make the unlearned information statistically indiscernible. Most importantly, the algorithm requires neither parametric fine-tuning nor in-context prompt calibration. A theoretical analysis demonstrates that this method satisfies rigorous privacy and unlearning guarantees. Additionally, experiments on real-world datasets prove that the method is both effective and has the capacity to handle practical unlearning tasks for large language model applications.

Deepfake techniques can now generate multimodal content comprising video and audio tracks. Compared with unimodal Deepfake images, videos or audio, multimodal Deepfake content is more deceptive and easily leads to the dissemination of hate speech, incitement to violence, and disinformation. Therefore, the detection of multimodal Deepfake has attracted much research attention recently. While cross-attention shows the promising capacity for modelling the complicated dependencies between audio and video in multimodal Deepfake detection, it fails to learn accurate cross-modal patterns if audio and video are misaligned in the temporal dimension. Besides, most current multimodal Deepfake detectors only provide a binary classification label, lacking fine-grained localization to identify significant forgery in multiple dimensions (e.g., modal, time, and spatial dimension). In this study, we propose a novel multimodal Deepfake detection framework named ForgeFinder, which goes beyond binary label prediction and achieves multi-grained forgery localization in modal and spatiotemporal dimensions. ForgeFinder incorporates both intra-modal and cross-modal inconsistencies to classify multimodal input. In detail, we adopt Serial Spatiotemporal Self-Attention (SSTSA) in the Intra-Modal Inconsistency Explorer (Intra-MIE), which allows the temporal self-attention to run in the original dimension without bringing unacceptable computational complexity. In the Cross-Modal Inconsistency Explorer (Cross-MIE), we propose the Offset-Shifted Cross-Attention (OSCA) by introducing a time offset term to the conventional cross-attention to mitigate the inaccuracy of cross-modal dependencies modelling brought by the temporal misalignment. By adopting the outputs of Intra-MIE for unimodal tasks, we identify the likelihood of modals being manipulated and localize tampered modals. At the same time, the attention weights of SSTSA can be visualized to pinpoint the temporal and spatial distribution of Deepfake manipulation. Therefore, for a single audio–video input sample, ForgeFinder not only tells the authenticity of the overall input but also localizes the modal, temporal sequence, and spatial coordinates of significant forgery, significantly contributing to more comprehensive forensics analysis. The results of extensive experiments indicate that ForgeFinder achieves state-of-the-art detection performance as well as accurate forgery localization in modal and spatiotemporal dimensions. Furthermore, experiments on content generated by Diffusion Models (DMs) show that our model also effectively recognizes DM-generated content.

The integration of AI with medical images enables the extraction of implicit image-derived biomarkers for a precise health assessment. Recently, retinal age, a biomarker predicted from fundus images, has become a proven predictor of systemic disease risks, behavioral patterns, aging trajectory and even mortality. However, the capability to infer such sensitive biometric data raises significant privacy risks, where unauthorized use of fundus images could lead to bioinformation leakage, breaching individual privacy. In response, we formulate a new research problem of biometric privacy associated with medical images and propose RetinaGuard, a novel privacy-enhancing framework that employs a feature-level generative adversarial masking mechanism to obscure retinal age while preserving image visual quality and disease diagnostic utility. The framework further utilizes a novel multiple-to-one knowledge distillation strategy incorporating a retinal foundation model and diverse surrogate age encoders to enable a universal defense against black-box age prediction models. Comprehensive evaluations confirm that RetinaGuard successfully obfuscates retinal age prediction with minimal impact on image quality and pathological feature representation. RetinaGuard is also flexible for extension to other medical image-derived biomarkers.

3D cars are commonly used in self-driving systems, virtual/augmented reality, and games. However, existing 3D car datasets are either synthetic or low-quality, limiting their applications in practical scenarios and presenting a significant gap toward high-quality real-world 3D car datasets. In this paper, we propose the first large-scale 3D real car dataset, termed 3DRealCar, offering three distinctive features. (1) \textbf{High-Volume}: 2,500 cars are meticulously scanned by smartphones, obtaining car images and point clouds with real-world dimensions; (2) \textbf{High-Quality}: Each car is captured in an average of 200 dense, high-resolution 360-degree RGB-D views, enabling high-fidelity 3D reconstruction; (3) \textbf{High-Diversity}: The dataset contains various cars from over 100 brands, collected under three distinct lighting conditions, including reflective, standard, and dark. Additionally, we offer detailed car parsing maps for each instance to promote research in car parsing tasks. Moreover, we remove background point clouds and standardize the car orientation to a unified axis for the reconstruction only on cars and controllable rendering without background. We benchmark 3D reconstruction results with state-of-the-art methods across different lighting conditions in 3DRealCar. Extensive experiments demonstrate that the standard lighting condition part of 3DRealCar can be used to produce a large number of high-quality 3D cars, improving various 2D and 3D tasks related to cars. Notably, our dataset brings insight into the fact that recent 3D reconstruction methods face challenges in reconstructing high-quality 3D cars under reflective and dark lighting conditions. \textcolor{red}{\href{https://xiaobiaodu.github.io/3drealcar/}{Our dataset is here.}}

Machine unlearning aims to remove the influence of specific samples from a trained model. A key challenge in this process is over-unlearning, where the model's performance on the remaining data significantly drops due to the change in the model's parameters. Existing unlearning algorithms depend on the remaining data to prevent this issue. As such, these methods are inapplicable in a more practical scenario, where only the unlearning samples are available (i.e., zero-shot unlearning). This paper presents a novel framework, ZS-PAG, to fill this gap. Our approach offers three key innovations: (1) we approximate the inaccessible remaining data by generating adversarial samples; (2) leveraging the generated samples, we pinpoint a specific subspace to perform the unlearning process, therefore preventing over-unlearning in the challenging zero-shot scenario; and (3) we consider the influence of the unlearning process on the remaining samples and design an influence-based pseudo-labeling strategy. As a result, our method further improves the model's performance after unlearning. The proposed method holds a theoretical guarantee, and experiments on various benchmarks validate the effectiveness and superiority of our proposed method over several baselines.

Federated Learning (FL) is a distributed paradigm aimed at protecting participant data privacy by exchanging model parameters to achieve high-quality model training. However, this distributed nature also makes FL highly vulnerable to backdoor attacks. Notably, the recently proposed state-of-the-art (SOTA) attack, 3DFed (SP2023), uses an indicator mechanism to determine whether the backdoor models have been accepted by the defender and adaptively optimizes backdoor models, rendering existing defenses ineffective. In this paper, we first reveal that the failure of existing defenses lies in the employment of empirical statistical measures that are loosely coupled with backdoor attacks. Motivated by this, we propose a Malignity-Aware backdooR defenSe (MARS) that leverages backdoor energy (BE) to indicate the malicious extent of each neuron. To amplify malignity, we further extract the most prominent BE values from each model to form a concentrated backdoor energy (CBE). Finally, a novel Wasserstein distance-based clustering method is introduced to effectively identify backdoor models. Extensive experiments demonstrate that MARS can defend against SOTA backdoor attacks and significantly outperforms existing defenses.

With the rapid adoption of LLM-based code assistants to enhance programming experiences, concerns over extraction attacks targeting private training data have intensified. These attacks specifically aim to extract Personal Information (PI) embedded within the training data of code generation models (CodeLLMs). Existing methods, using either manual or semi-automated techniques, have successfully extracted sensitive data from these CodeLLMs. However, the limited amount of data currently retrieved by extraction attacks risks significantly underestimating the true extent of training data leakage. In this paper, we propose an automatic PI data extraction attack framework against LLM-based code assistants, named Codebreaker. This framework is built on two core components: (i) the introduction of semantic entropy, which evaluates the likelihood of a prompt triggering the model to respond with training data; and (ii) an automatic dynamic mutation mechanism that seamlessly integrates with Codebreaker, reinforcing the iterative process across the framework and promoting greater interconnection between different PI elements within a single response. This boosts reasoning diversity, model memorization, and finally attack performance. Using six series of open-source CodeLLMs (i.e., CodeParrot, StarCoder2, Code Llama, CodeGemma, DeepSeek-Coder, DeepSeek-V3) and two commercial code assistants (i.e., CodeFuse and GPT), we demonstrate the effectiveness of our proposed framework: (i) Codebreaker outperforms all current state-of-the-art extraction attacks by 6.22% ~ 44.9% (averaging 21.79%); (ii) when PI within a single response originates from the same GitHub repository, our framework - considering multiple interconnections in the response - exceeds others by 3.88% ~ 32.37% (averaging 15.31%). Furthermore, we discuss potential defenses, highlighting the urgent need for stronger measures to prevent PI leakage at the base model level.

Misinformation is a significant societal issue with potentially severe consequences. It appears in text, image, audio, and video modalities, encompassing various categories such as unimodal deception (fact-conflicting, AI-generated & offensive content) and cross-modal inconsistencies. However, current detection approaches often focus on text and image, overlooking the growing prevalence of misinformation in audio and video content. Moreover, these methods typically tend to address only one or two types of misinformation, failing to address all categories simultaneously. These detectors are also usually designed to make judgments without providing explanations, reducing transparency and limiting their broader applicability. To address these issues, we propose MDAM3, a Misinformation Detection and Analysis Framework for Multitype Multimodal Media. MDAM3 analyzes each input in internal detection and examines relationships across modalities to identify inconsistencies. It utilizes web resources and integrates Large Vision-Language Models (LVLMs) to deliver accurate detection results along with detailed analysis. To evaluate MDAM3, we curate MDAM3-DB, a specialized multitype multimodal misinformation dataset. A user study is conducted to explore MDAM3's usability, interpretability, and effectiveness. We hope this research contributes to advancing misinformation detection methodologies and provides valuable insights for developing robust multimodal analysis tools.

Abstract Artificial intelligence (AI) is revolutionizing both industries and reshaping the global economy. However, the rapid advancement of AI technologies brings significant security and privacy challenges. Recent incidents highlight vulnerabilities in AI systems, such as data leakage and malicious code injection, leading to severe financial losses and privacy breaches. Although existing studies have discussed specific security threats, they often lack detailed granularity and cover a limited scope. In this survey, we fill this gap by systematically categorizing and analyzing the threats and countermeasures in AI systems, which span both the training and inference stages, encompass centralized and distributed settings, and address both conventional and foundation AI models. By reviewing existing literature, we aim to provide AI researchers and practitioners with a thorough understanding of system vulnerabilities and current countermeasures. We hope to inspire further research into robust solutions, ultimately contributing to the development of resilient AI technologies.

Deepfake technology, leveraging advanced artificial intelligence (AI) algorithms, has emerged as a powerful tool for generating hyper‐realistic synthetic human faces, presenting both innovative opportunities and significant challenges. Meanwhile, the development of Deepfake detectors represents another branch of models striving to recognize AI‐generated fake faces and protect people from the misinformation of Deepfake. This ongoing cat‐and‐mouse game between generation and detection has spurred a dynamic evolution in the landscape of Deepfake. This survey comprehensively studies recent advancements in Deepfake generation and detection techniques, focusing particularly on the utilization of generative adversarial networks (GANs) and diffusion models (DMs). For both GAN‐based and DM‐based Deepfake generators, we categorize them based on whether they synthesize new content or manipulate existing content. Correspondingly, we examine various strategies employed to identify synthetic and manipulated Deepfake, respectively. Finally, we summarize our findings by discussing the unique capabilities and limitations of GANs and DM in the context of Deepfake. We also identify promising future directions for research, including the development of hybrid approaches that leverage the strengths of both GANs and DM, the exploration of novel detection strategies utilizing advanced AI techniques, and the ethical considerations surrounding the development of Deepfake. This survey paper serves as a valuable resource for researchers, practitioners, and policymakers seeking to understand the state‐of‐the‐art in Deepfake technology, its implications, and potential avenues for future research and development.

Federated learning (FL) is a distributed and privacy‐preserving machine learning paradigm that coordinates multiple clients to train a model while keeping the raw data localized. However, this traditional FL poses some challenges, including privacy risks, data heterogeneity, communication bottlenecks, and system heterogeneity issues. To tackle these challenges, knowledge distillation (KD) has been widely applied in FL since 2020. KD is a validated and efficacious model compression and enhancement algorithm. The core concept of KD involves facilitating knowledge transfer between models by exchanging logits at intermediate or output layers. These properties make KD an excellent solution for the long‐lasting challenges in FL. Up to now, there have been few reviews that summarize and analyze the current trend and methods for how KD can be applied in FL efficiently. This article aims to provide a comprehensive survey of KD‐based FL, focusing on addressing the above challenges. First, we provide an overview of KD‐based FL, including its motivation, basics, taxonomy, and a comparison with traditional FL and where KD should execute. We also analyze the critical factors in KD‐based FL in the Appendix, including teachers, knowledge, data, and methods. We discuss how KD can address the challenges in FL, including privacy protection, data heterogeneity, communication efficiency, and personalization. Finally, we discuss the challenges facing KD‐based FL algorithms and future research directions. We hope this survey can provide insights and guidance for researchers and practitioners in the FL area.

With the rapid advancement of the Internet of Things (IoT), its applications are becoming increasingly essential in actual application. Specifically, the recent surge in electric vehicles has spurred significant advancements in vehicle ad hoc networks (VANET). Therefore, efficient data transmission is a critical challenge in IoT, especially VANET. The certificateless signcryption (CLS) scheme has high authentication efficiency, which has become increasingly important in IoT. However, most existing schemes rely on bilinear pairing, resulting in high calculation costs or failure to achieve their claimed security. Furthermore, the security of some CLS schemes is proved in the random oracle, which limits the usefulness of the CLS scheme. To further address these issues, we propose an efficient and secure CLS scheme in the standard model for VANET, which enhances its practical applicability in VANET. Our construction employs formal security proofs and cost simulations to ensure the scheme’s security and low calculation costs, making it suitable for VANET. Finally, the security and efficiency analyses prove that our scheme offers enhanced security features and superior performance compared to existing schemes.

The Industrial Internet of Things (IIoT) is changing our way of life and work. As the number of mobile devices connected to IIoT increases, users will face many security challenges, such as insecure communication environments. The certificateless signature (CLS) scheme can ensure the integrity and validity of data and provide secure identity authentication for the IIoT, and several pairing-free CLS schemes have been proposed in recent years. However, we find they are vulnerable to the signature forgery attack of malicious key generation centers by safety analysis, which does not realize the security they have claimed. To solve this problem, we show an improved CLS scheme that achieves anonymity and formally proves its security under the hardness of the discrete logarithm problem in the random oracle. Comprehensive performance analysis and comparison show that our scheme has less communication and computation costs with higher security, and our construction is suitable for scenarios with limited resources. Finally, we apply this scheme to construct a mutual identity authentication protocol in the healthcare IIoT environment.

With growing attention to data privacy in federated learning, federated unlearning has become an important solution to meet increasing demands for privacy compliance. However, unlearning may bring in new security concerns, such as dangers of adversarial manipulation, where the adversary may launch malicious updates or inputs to hurt the model performance or prediction, privacy-attacks, as the sensitive data can be possibly deduced from the process of unlearning, and performance degradation, because the unlearning process may break the consistency or performance of the model. In this paper, to address such issues and acquire a good and adaptive unlearning policy without causing much negative effect to the federated system, we present a reinforcement learning based method to facilitate the data unlearning method in federated learning. Our approach iteratively disposes of clients through partial unlearning, complete unlearning, or no unlearning using a DQN combined with clients’ properties like contribution, privacy cost, and computational overhead. We show that by utilizing the reinforcement learning technique, the performance decay can be defended effectively, and adversarial behaviors are indeed a common concern for the federated unlearning scenario. Our analysis can inform the development of federated unlearning frameworks that defend against performance and security threats.

Quantum computing offers unparalleled processing power but raises significant data privacy challenges.Quantum Differential Privacy(QDP) leverages inherent quantum noise to safeguard privacy, surpassing traditional DP. This paper develops comprehensive noise profiles, identifies noise types beneficial for QDP, and highlights the need for practical implementations beyond theoretical models. Existing QDP mechanisms, limited to single noise sources, fail to reflect the multi-source noise reality of quantum systems. We propose a resilient hybrid QDP mechanism utilizing channel and measurement noise, optimizing privacy budgets to balance privacy and utility. Additionally, we introduceLifted Quantum Differential Privacy, offering enhanced randomness for improved privacy audits and quantum algorithm evaluation.

Accurate detection and subsequent cleanup of floating waste are critical for ecosystem protection. However, existing methods face significant challenges in dealing with unpredictable weather, limiting their generalization capabilities. This paper proposes a novel plug-and-play architecture called Image Enhancer with Spatial Search and Aggregation (SpatialIE) . This architecture includes a novel encoder block (KANsformer) based on KANs and a dynamic decoder module leveraging prompt-based learning. The KANsformer in the encoder enhances the model’s ability to learn complex non-linear relationships in degraded images . At the same time, the Multi-Order-Based Prompt Block in the decoder enables dynamic optimization of feature processing strategies. These components allow SpatialIE to refine detection strategies to accommodate unknown weather conditions adaptively. We integrate SpatialIE with common YOLO detectors in an end-to-end training framework using the FloW-img and Water Surface Object Detection Dataset (WSODD) datasets. Experimental results show that our method delivers outstanding performance across multiple and single degradation scenarios, achieving an optimal balance between detection accuracy and storage efficiency.

Electrocardiogram(ECG)-based biometric authentication has become a promising method to improve security in wearable devices due to its inherent uniqueness and difficulty to replicate. However, no studies currently demonstrate that ECG authentication can resist modern attack techniques employed against biometric authentication. In this paper, we present a novel dictionary attack against ECG authentication systems, which poses a significant threat. In contrast to conventional targeted attacks, this approach utilizes random pairing to breach a vast number of users, without requiring specific information about their biometric data. Our approach leverages adversarial optimization and clustering to generate synthetic ECG waveforms capable of bypassing authentication mechanisms of various systems, revealing critical vulnerabilities in the current implementation of ECG-based biometrics. We comprehensively evaluate the effectiveness of this attack across different ECG authentication models, demonstrating that despite the intrinsic uniqueness of ECG signals, a substantial number of users are vulnerable. Our attack method can bypass the authentication system of an average of 20% of users even at the most stringent false acceptance rate of 1%. With up to five attack attempts allowed, our method can bypass up to 62% of users’ ECG authentication models.

Fairness has emerged as a crucial concern in machine learning since biased models would generate dissimilar predictions for different groups, perpetuating social inequalities. Although numerous techniques have been proposed to address the fairness issue in machine learning, most rely on incorporating fairness constraints during the training phase, rendering them ineffective once the model is deployed. This paper explores the potential of fine-tuning biased models to enhance fairness, particularly suitable for scenarios where retraining the model is not feasible. Our approach is rooted in an empirical analysis of the distribution of bias within a biased model, and we fine-tune the model parameter in a limited scope so that the performance of the original model can be maintained. We first observe that fine-tuning a biased model leads to deviations from its initial state, with deep layers undergoing the most significant changes. We then design and apply a bias-discovery algorithm, revealing that bias predominantly resides in the model’s deep layers. Based on these observations, we propose a straightforward yet highly effective method for debiasing the model: fine-tuning the classification head. We conduct a thorough theoretical analysis to justify the proposed method and provide guidance for fine-tuning. Furthermore, we experimentally validate our method on tabular and image datasets using four networks (CNN, AlexNet, VGG-11, and ResNet-18).

Graph data contains rich node features and unique edge information, which have been applied across various domains, such as citation networks or recommendation systems. Graph Neural Networks (GNNs) are specialized for handling such data and have shown impressive performance in many applications. However, GNNs may contain of sensitive information and susceptible to privacy attacks. For example, link stealing is a type of attack in which attackers infer whether two nodes are linked or not. Previous link stealing attacks primarily relied on posterior probabilities from the target GNN model, neglecting the significance of node features. Additionally, variations in node classes across different datasets lead to different dimensions of posterior probabilities. The handling of these varying data dimensions posed a challenge in using a single model to effectively conduct link stealing attacks on different datasets. To address these challenges, we introduce Large Language Models (LLMs) to perform link stealing attacks on GNNs. LLMs can effectively integrate textual features and exhibit strong generalizability, enabling attacks to handle diverse data dimensions across various datasets. We design two distinct LLM prompts to effectively combine textual features and posterior probabilities of graph nodes. Through these designed prompts, we fine-tune the LLM to adapt to the link stealing attack task. Furthermore, we fine-tune the LLM using multiple datasets and enable the LLM to learn features from different datasets simultaneously. Experimental results show that our approach significantly enhances the performance of existing link stealing attack tasks in both white-box and black-box scenarios. Our method can execute link stealing attacks across different datasets using only a single model, making link stealing attacks more applicable to real-world scenarios.

Graph neural networks (GNNs) have obtained considerable attention due to their ability to leverage the inherent topological and node information present in graph data. While extensive research has been conducted on privacy attacks targeting machine learning models, the exploration of privacy risks associated with node-level membership inference attacks on GNNs remains relatively limited. GNNs learn representations that encapsulate valuable information about the nodes. These learned representations can be exploited by attackers to infer whether a specific node belongs to the training dataset, leading to the disclosure of sensitive information. The insidious nature of such privacy breaches often leads to an underestimation of the associated risks. Furthermore, the inherent challenges posed by node membership inference attacks make it difficult to develop effective attack models for GNNs that can successfully infer node membership. We propose a more efficient approach that specifically targets node-level membership inference attacks on GNNs. Initially, we combine nodes and their respective neighbors to carry out node membership inference attacks. To address the challenge of variable-length features arising from the differing number of neighboring nodes, we introduce an effective feature processing strategy. Furthermore, we propose two strategies: multiple training of shadow models and random selection of non-membership data, to enhance the performance of the attack model. We empirically evaluate the efficacy of our proposed method using three benchmark datasets. Additionally, we explore two potential defense mechanisms against node-level membership inference attacks.

As an increasing number of data-driven deep learning models are deployed in our daily lives, the issue of algorithmic fairness has become a major concern. These models are trained on data that inevitably contains various biases, leading them to learn unfair representations that differ across demographic subgroups, resulting in unfair predictions. Previous work on fairness has attempted to remove subgroup information from learned features, aiming to contribute to similar representations across subgroups and lead to fairer predictions. However, identifying and removing this information is extremely challenging due to the “black box” nature of neural networks. Moreover, removing desired features without affecting other features is difficult, as features are often correlated, potentially harming model prediction performance. This paper aims to learn fair representations without degrading model prediction performance. We adopt knowledge distillation, allowing unfair models to learn fair representations directly from a fair teacher. The proposed method provides a novel approach to obtaining fair representations while maintaining valid prediction performance. We evaluate the proposed method, FairDistill, on four datasets (CIFAR-10, UTKFace, CelebA, and Adult) under diverse settings. Extensive experiments demonstrate the effectiveness and robustness of the proposed method.

Federated learning is a promising privacy-preserving paradigm for distributed machine learning. In this context, there is sometimes a need for a specialized process called machine unlearning, which is required when the effect of some specific training samples needs to be removed from a learning model due to privacy, security, usability, and/or legislative factors. However, problems arise when current centralized unlearning methods are applied to existing federated learning, in which the server aims to remove all information about a class from the global model. Centralized unlearning usually focuses on simple models or is premised on the ability to access all training data at a central node. However, training data cannot be accessed on the server under the federated learning paradigm, conflicting with the requirements of the centralized unlearning process. Additionally, there are high computation and communication costs associated with accessing clients’ data, especially in scenarios involving numerous clients or complex global models. To address these concerns, we propose a more effective and efficient federated unlearning scheme based on the concept of model explanation. Model explanation involves understanding deep networks and individual channel importance, so that this understanding can be used to determine which model channels are critical for classes that need to be unlearned. We select the most influential channels within an already-trained model for the data that need to be unlearned and fine-tune only influential channels to remove the contribution made by those data. In this way, we can simultaneously avoid huge consumption costs and ensure that the unlearned model maintains good performance. Experiments with different training models on various datasets demonstrate the effectiveness of the proposed approach.

Graph Neural Networks (GNNs), specifically designed to process the graph data, have achieved remarkable success in various applications. Link stealing attacks on graph data pose a significant privacy threat, as attackers aim to extract sensitive relationships between nodes (entities), potentially leading to academic misconduct, fraudulent transactions, or other malicious activities. Previous studies have primarily focused on single datasets and did not explore cross-dataset attacks, let alone attacks that leverage the combined knowledge of multiple attackers. However, we find that an attacker can combine the data knowledge of multiple attackers to create a more effective attack model, which can be referred to cross-dataset attacks. Moreover, if knowledge can be extracted with the help of Large Language Models (LLMs), the attack capability will be more significant. In this paper, we propose a novel link stealing attack method that takes advantage of cross-dataset and LLMs. The LLM is applied to process datasets with different data structures in cross-dataset attacks. Each attacker fine-tunes the LLM on their specific dataset to generate a tailored attack model. We then introduce a novel model merging method to integrate the parameters of these attacker-specific models effectively. The result is a merged attack model with superior generalization capabilities, enabling effective attacks not only on the attackers' datasets but also on previously unseen (out-of-domain) datasets. We conducted extensive experiments in four datasets to demonstrate the effectiveness of our method. Additional experiments with three different GNN and LLM architectures further illustrate the generality of our approach. In summary, we present a new link stealing attack method that facilitates collaboration among multiple attackers to develop a powerful, universal attack model that reflects realistic real-world scenarios.

Graph Neural Networks (GNNs), a type of machine learning model has been widely used in social networks, drug recommendations, and various other domains. While GNNs provide significant benefits, they also raise privacy concerns such as membership inference attack, posing a substantial risk to the private training data of GNNs. Previous studies have demonstrated that GNNs are susceptible to membership inference attacks, revealing private information about the training graph. However, these studies overlook the challenges of training data imbalance and a small number of training datasets in node-level membership inference attacks. Additionally, they under-utilize the topological features of the graph, resulting in sub-optimal performance of the attack models. In this paper, we find that by using attention mechanism, attackers have higher attack accuracy on the node-level membership inference attacks. This is because attention mechanism assigns different weights to neighboring nodes based on their contribution to the membership inference attack. To implement the attack, first, we tackle the training data imbalance issue through a random selection strategy. Second, we propose a data augmentation method exclusively tailored for membership inference attacks to address the problem of small datasets in node-level membership inference attacks. Next, we leverage the topological features of the graph and introduce two different feature padding strategies in feature processing. Finally, we employ the attention mechanism to assign different weights to neighboring nodes, enhancing the accuracy of the attack. We evaluate our proposed method on three datasets and three different GNN models, and the experimental results consistently demonstrate outstanding performance

As machine learning continues to develop and data misuse scandals become more prevalent, individuals are becoming increasingly concerned about their personal information and are advocating for the right to remove their data. Machine unlearning has emerged as a solution to erase training data from trained machine learning models. Despite its success in classifiers, research on Generative Adversarial Networks (GANs) is limited due to their unique architecture, including a generator and a discriminator. One challenge pertains to generator unlearning, as the process could potentially disrupt the continuity and completeness of the latent space. This disruption might consequently diminish the model's effectiveness after unlearning. Another challenge is how to define a criterion that the discriminator should perform for the unlearning images. In this paper, we introduce a substitution mechanism and define a fake label to effectively mitigate these challenges. Based on the substitution mechanism and fake label, we propose a cascaded unlearning approach for both item and class unlearning within GAN models, in which the unlearning and learning processes run in a cascaded manner. We conducted a comprehensive evaluation of the cascaded unlearning technique using the MNIST, CIFAR-10, and FFHQ datasets, analyzing its performance across four key aspects: unlearning effectiveness, intrinsic model performance, impact on downstream tasks, and unlearning efficiency. Experimental results demonstrate that this approach achieves significantly improved item and class unlearning efficiency, reducing the required time by up to$185\times$and$284\times$for the MNIST and CIFAR-10 datasets, respectively, in comparison to retraining from scratch. Notably, although the model's performance experiences minor degradation after unlearning, this reduction is negligible when dealing with a minimal number of images (e.g., 64) even for high-resolution FFHQ dataset and has no adverse effects on downstream tasks such as classification.

Generative Adversarial Network (GAN) models have shown promise in detecting adversarial examples for classifiers by leveraging their ability to reconstruct benign and adversarial examples differently. However, existing approaches often struggle with reconstruction optimization convergence for adversarial examples, as GANs cannot perfectly reconstruct such inputs. To address this instability, we propose two novel methods: Average-RAED and Strict-RAED. Average-RAED focuses on the average reconstruction performance throughout the optimization process, with the difference in the classifier’s logit information between the candidate sample and its reconstructed version as the detection signal. This approach enhances robustness and effectiveness, particularly for complex datasets. Strict-RAED integrates classification information into the reconstruction process and imposes a strict constraint to keep the latent code within the initial latent subspace. This design enables Strict-RAED to effectively identify both manually-crafted and natural adversarial examples. We evaluated both methods in a grey-box scenario, where the adversary has full knowledge of the target classifier’s architecture and parameters but no information about the defense GAN. Our results demonstrate that both Average-RAED and Strict-RAED outperform baseline methods in distinguishing adversarial examples from benign samples. Notably, Strict-RAED achieves higher detection rates for both manually-crafted and natural adversarial examples. Furthermore, combining Strict-RAED for detection and Average-RAED for reforming candidate samples significantly improves classifier accuracy against less significant adversarial examples.

While in-processing fairness approaches show promise in mitigating biased predictions, their potential impact on privacy leakage remains under-explored. We aim to address this gap by assessing the privacy risks of fairness-enhanced binary classifiers via membership inference attacks (MIAs) and attribute inference attacks (AIAs). Surprisingly, our results reveal that enhancing fairness does not necessarily lead to privacy compromises. For example, these fairness interventions exhibit increased resilience against MIAs and AIAs. This is because fairness interventions tend to remove sensitive information among extracted features and reduce confidence scores for the majority of training data for fairer predictions. However, during the evaluations, we uncover a potential threat mechanism that exploits prediction discrepancies between fair and biased models, leading to advanced attack results for both MIAs and AIAs. This mechanism reveals potent vulnerabilities of fair models and poses significant privacy risks of current fairness methods. Extensive experiments across multiple datasets, attack methods, and representative fairness approaches confirm our findings and demonstrate the efficacy of the uncovered mechanism. Our study exposes the under-explored privacy threats in fairness studies, advocating for thorough evaluations of potential security vulnerabilities before model deployments.

Text-to-image generative models have gained popularity among both researchers and the general public due to their ability to generate high-quality images from text prompts. However, many of these models rely on pre-trained text encoders obtained from external sources, which introduces significant security risks. Adversaries can implant backdoors into these encoders, causing the models to generate images with predefined attributes, such as nudity, or to produce malicious content when triggered prompts are used. Existing backdoor defenses are ineffective in text-to-image models due to the large number of model parameters, the difficulty of collecting high-quality text-image pairs, and the challenges in defining adversarial behaviors. In this paper, we propose two defense methods: few-shot fine-tuning and history-based fine-tuning. Specifically, few-shot fine-tuning eliminates backdoors by adding words that present backdoor effects to the input prompts to make the embeddings of trigger prompts consistent with those of clean prompts. History-based fine-tuning distinguishes triggered prompts from clean prompts by removing keywords from the input prompts and utilizes only historical data for self-healing. Experimental results demonstrate that both defense methods can effectively eliminate backdoors while maintaining high model performance. Our code is available at https://github.com/Wu-sm/Defense-against-backdoor-attacks-in-text-to-image.

Machine unlearning enables pre-trained models to remove the effect of certain portions of training data. Previous machine unlearning schemes have mainly focused on unlearning a cluster of instances or all instances belonging to a specific class. These types of unlearning might have a significant impact on the model utility; and they may be inadequate for situations where we only need to unlearn features within instances, rather than the whole instances. Due to the different granularity, current unlearning methods can hardly achieve feature-level unlearning. To address the challenges of utility and granularity, we propose a refined granularity unlearning scheme referred to as “feature unlearning”. We first explore two distinct scenarios based on whether the annotation information about the features is given: feature unlearning with known annotations and feature unlearning without annotations. Regarding unlearning with known annotations, we propose an adversarial learning approach to automatically remove effects about features. For unlearning without annotations, we initially enable the output of one model's layer to identify different pattern features using model interpretability techniques. We proceed to filter features from instances based on these outputs with identifying ability. So that we can remove the feature impact based on filtered instances and the fine-tuning process. The effectiveness of our proposed approach is demonstrated through experiments involving diverse models on various datasets in different scenarios.

Transfer learning is an important approach that produces pre-trained teacher models which can be used to quickly build specialized student models. However, recent research on transfer learning has found that it is vulnerable to various attacks, e.g., misclassification and backdoor attacks. However, it is still not clear whether transfer learning is vulnerable to model inversion attacks. Launching a model inversion attack against transfer learning scheme is challenging. Not only does the student model hide its structural parameters, but it is also not queried to the adversary. Hence, when targeting a student model, existing model inversion attacks fail, as they typically rely on querying the target model. In this paper, we initiate research into model inversion attacks against transfer learning with two novel attack methods. Both are black-box attacks, suiting different situations, that do not rely on queries to the target student model. In the first method, the adversary has the data samples that share the same distribution as the training set of the teacher model. In the second method, the adversary does not have any such samples. Experiments show that highly recognizable data records can be inverted with both of these methods. This research underscores the critical insight that even when a model is shielded from public queries, it can still be susceptible to model inversion attacks.

Federated learning (FL) trains a global model by aggregating local updates from multiple clients under a server's guidance. Despite its potential, FL is vulnerable to poisoning attacks where malicious clients intentionally corrupt their updates, compromising the global model's accuracy. Current defense strategies aim to tolerate or remove such corrupt updates, but they are not fully effective to prevent malicious clients from sending poisonous updates to the server, leaving the global model at risk. We propose a novel approach based on the evolution of cooperation, which promotes system-wide collaboration. Our defense method allows the server to selectively engage clients in the training process, encouraging them to provide clean updates or exclude those persistently malicious. We also introduce an attack framework where clients initially send clean updates to gain trust before sending malicious ones later. This model, designed to simulate advanced threats, can adapt to various attack types to increase its impact. Our experimental results show that this defense significantly improves resilience against such attacks, effectively safeguarding the global model even under complex threat scenarios.

The Right to be Forgottenand related legislation mandate that every individual has the right to withdraw their consent to the use of their personal data. These laws and regulations led to the development of a new concept – machine unlearning. Recently, various machine unlearning approaches have been proposed to remove the influence of data sample(s) from a trained model. However, alongside all the achievements, there are still loopholes that may cause significant losses for the model owner. We found that it might be dangerous to remove the influence of some “well-designed” data samples from a pre-trained model. An adversary may craft some poisonous data and take advantage of the unlearning request to manipulate an unlearned model. In this paper, we exploit the vulnerabilities of the fast retraining-based unlearning strategy and propose a new data poisoning attack to demonstrate the risks they may bring about. The attack is triggered by submitting an unlearning request on adversarially designed data samples, and it enables an adversary to steer a model to a target location in the parameter space. We provide theoretical analyses to show the feasibility of such an attack regarding the gradient of the poisonous data. Furthermore, the experimental results on real-world datasets demonstrate that the attack strategy is effective.

Applying differential privacy (DP) to federated learning (FL) effectively safeguards participants’ training data against privacy threats, yet the stringent availability requirements of FL present a significant challenge in optimizing accuracy while ensuring privacy. Integrating differential privacy continuous data release (DPCR) into private FL mitigates the errors accumulating on intermediate parameter models securely, thereby enhancing accuracy, with performance gains driven by developing advanced DPCR models. In this context, we propose ak-ary Tree-based DPCR (kTCR) model to provide deeper and more flexible error optimization, thereby promoting a robust accuracy enhancement for private FL. Our kTCR model introduces Variance Optimal Estimation (VOE) and privacy budget allocation (PBA) methods to optimize accuracy, posing significant efficiency challenges simultaneously. With rigorous mathematical analysis, we reduce the computational complexity of VOE fromO(t3) toO(lgt), then introduce a meta-factor method that transforms the challenging PBA issues into a convex optimization problem with significantly reduced variables (e.g., a 3-ary tree with 5.23 × 109nodes requiring only 61 variables), thus yielding high efficiencies. Our experiments on classical datasets demonstrate that our kTCR model with appropriatekoutperforms the state-of-the-art ABCRG by 0.7% ~ 2.0% in accuracy and traditional private FL by 5.5% ~ 18.7%. Further experiments demonstrate that adjusting the aritykeffectively reduces the Pre-aggregation Error, leading to a further 2.09% accuracy gain. Our achievements both improve the accuracy of private FL and provide new insights into building high-availability FL with DP.

Model extraction attacks currently pose a non-negligible threat to the security and privacy of deep learning models. By querying the model with a small dataset and using the query results as the ground-truth labels, an adversary can steal a piracy model with performance comparable to the original model. Two key issues that cause the threat are, on the one hand, accurate and unlimited queries can be obtained by the adversary; on the other hand, the adversary can aggregate the query results to train the model step by step. The existing defenses usually employ model watermarking or fingerprinting to protect the ownership. However, these methods cannot proactively prevent the violation from happening. To mitigate the threat, we propose QUEEN (QUEry unlEarNing) that proactively launches counterattacks on potential model extraction attacks from the very beginning. To limit the potential threat, QUEEN has sensitivity measurement and outputs perturbation that prevents the adversary from training a piracy model with high performance. In sensitivity measurement, QUEEN measures the single query sensitivity by its distance from the center of its cluster in the feature space. To reduce the learning accuracy of attacks, for the highly sensitive query batch, QUEEN applies query unlearning, which is implemented by gradient reverse to perturb the softmax output such that the piracy model will generate reverse gradients to worsen its performance unconsciously. Experiments show that QUEEN outperforms the state-of-the-art defenses against various model extraction attacks with a relatively low cost to the model accuracy. The artifact is publicly available athttps://github.com/MaraPapMann/QUEEN.

Social engineering malware, which exploits both technical and human vulnerabilities, presents challenging for individuals and organizations. However, existing studies typically focus on either technical or human vulnerabilities through case studies or questionnaires, ignoring their combined importance in mitigating such threats. This study pioneers the introduction of a mathematical model to analyze and mitigate the dynamics associated with these combined vulnerabilities. To achieve this, this study proposes an innovative framework, which integrates (a) acoupled malware-opinion dynamics modelto capture the interplay between both types of vulnerabilities, and (b) anoptimal impulse control approachto strategically mitigatingsocial engineering malware. Within this framework, we define an optimization problem, aimed at balancing control costs and malware severity. We derive theoretical conditions for optimal impulse strategies that achieve this balance and develop an iterative algorithm, the convergence and scalability of which have been empirically validated. Experimental results on three real-world social networks and synthetic scale-free networks demonstrate that our strategies consistently achieve an optimal balance by minimizing total expenses, including control costs and losses associated with malware. This finding underscores the effectiveness of routine patching and ongoing security awareness training in standard cybersecurity practices. Further experiments indicate that the strategic, early, and frequent deployment of patches in specific scenarios can effectively reduce unnecessary losses, enhancing overall cybersecurity resilience.

With the extensive use of machine learning technologies, data providers encounter increasing privacy risks. Recent legislation, such as GDPR, obligates organizations to remove requested data and its influence from a trained model. Machine unlearning is an emerging technique designed to enable machine learning models to erase users’ private information. Although several efficient machine unlearning schemes have been proposed, these methods still have limitations. First, removing the contributions of partial data may lead to model performance degradation. Second, discrepancies between the original and generated unlearned models can be exploited by attackers to obtain target sample’s information, resulting in additional privacy leakage risks. To address above challenges, we proposed a game-theoretic machine unlearning algorithm that simulates the competitive relationship between unlearning performance and privacy protection. This algorithm comprises unlearning and privacy modules. The unlearning module possesses a loss function composed of model distance and classification error, which is used to derive the optimal strategy. The privacy module aims to make it difficult for an attacker to infer membership information from the unlearned data, thereby reducing the privacy leakage risk during the unlearning process. Additionally, the experimental results on real-world datasets demonstrate that this game-theoretic unlearning algorithm’s effectiveness and its ability to generate an unlearned model with a performance similar to that of the retrained one while mitigating extra privacy leakage risks.

Federated learning (FL) has gained significant attention as it facilitates collaborative machine learning among multiple clients without centralizing their data on a server. FL ensures the privacy of participating clients by locally storing their data, which creates new challenges in fairness. Traditional debiasing methods assume centralized access to sensitive information, rendering them impractical for the FL setting. Additionally, FL is more susceptible to fairness issues than centralized machine learning due to the diverse client data sources that may be associated with group information. Therefore, training a fair model in FL without access to client local data is important and challenging. This article presents AFed, a straightforward, yet effective framework for promoting group fairness in FL. The core idea is to circumvent restricted data access by learning the global data distribution. This article proposes two approaches: AFed-G, which uses a conditional generator trained on the server side, and AFed-GAN, which improves upon AFed-G by training a conditional GAN on the client side. We augment the client data with the generated samples to help remove bias. Our theoretical analysis justifies the proposed methods, and empirical results on multiple real-world datasets demonstrate a substantial improvement in AFed over several baselines.

While existing fairness interventions show promise in mitigating biased predictions, most studies concentrate on single-attribute protections. Although a few methods consider multiple attributes, they either require additional constraints or prediction heads, incurring high computational overhead or jeopardizing the stability of the training process. More critically, they consider per-attribute protection approaches, raising concerns about fairness gerrymandering where certain attribute combinations remain unfair. This work aims to construct a neutral domain containing fused information across all subgroups and attributes. It delivers fair predictions as the fused input contains neutralized information for all considered attributes. Specifically, we adopt mixup operations to generate samples with fused information. However, our experiments reveal that directly adopting the operations leads to degraded prediction results. The excessive mixup operations result in unrecognizable training data. To this end, we design three distinct mixup schemes that balance information fusion across attributes while retaining distinct visual features critical for training valid models. Extensive experiments with multiple datasets and up to eight sensitive attributes demonstrate that the proposed MultiFair method can deliver fairness protections for multiple attributes while maintaining valid prediction results.

Machine unlearning is an emerging technology that has come to attract widespread attention. A number of factors, including regulations and laws, privacy, and usability concerns, have resulted in this need to allow a trained model to forget some of its training data. Existing studies of machine unlearning mainly focus on unlearning requests that forget a cluster of instances or all instances from one class. While these approaches are effective in removing instances, they do not scale to scenarios where partial targets within an instance need to be forgotten. For example, one would like to only unlearn a person from all instances that simultaneously contain the person and other targets. Directly migrating instance-level unlearning to target-level unlearning will reduce the performance of the model after the unlearning process, or fail to erase information completely. To address these concerns, we have proposed a more effective and efficient unlearning scheme that focuses on removing partial targets from the model, which we name "target unlearning." Specifically, we first construct an essential graph data structure to describe the relationships between all important parameters that are selected based on the model explanation method. After that, we simultaneously filter parameters that are also important for the remaining targets and use the pruning-based unlearning method, which is a simple but effective solution to remove information about the target that needs to be forgotten. Experiments with different training models on various datasets demonstrate the effectiveness of the proposed approach.

Model inversion attacks pose a significant privacy threat to machine learning models by reconstructing sensitive data from their outputs. While various defenses have been proposed to counteract these attacks, they often come at the cost of the classifier's utility, thus creating a challenging trade-off between privacy protection and model utility. Moreover, most existing defenses require retraining the classifier for enhanced robustness, which is impractical for large-scale, well-established models. This article introduces a novel defense mechanism to better balance privacy and utility, particularly against adversaries who employ a machine learning model (i.e., inversion model) to reconstruct private data. Drawing inspiration from data poisoning attacks, which can compromise the performance of machine learning models, we propose a strategy that leverages data poisoning to contaminate the training data of inversion models, thereby preventing model inversion attacks. Two defense methods are presented. The first, termed label-preserving poisoning attacks for all output vectors (LPA), involves subtle perturbations to all output vectors while preserving their labels. Our findings demonstrate that these minor perturbations, introduced through a data poisoning approach, significantly increase the difficulty of data reconstruction without compromising the utility of the classifier. Subsequently, we introduce a second method, label-flipping poisoning for partial output vectors (LFP), which selectively perturbs a small subset of output vectors and alters their labels during the process. Empirical results indicate that LPA is notably effective, outperforming the current state-of-the-art defenses. Our data poisoning-based defense provides a new retraining-free defense paradigm that preserves the victim classifier's utility.

Machine unlearning is an emerging paradigm that aims to make machine learning models “forget” what they have learned about particular data. It fulfills the requirements of privacy legislation (e.g., General Data Protection Regulation), which stipulates that individuals have the autonomy to determine the usage of their personal data. However, alongside all the achievements, there are still loopholes in machine unlearning that may cause significant losses for the system, especially in edge computing. Edge computing is a distributed computing paradigm with the purpose of migrating data–processing tasks closer to terminal devices. While various machine unlearning approaches have been proposed to erase the influence of data sample(s), we claim that it might be dangerous to directly apply them in the realm of edge computing. A malicious edge node may broadcast (possibly fake) unlearning requests to a target data sample (s) and then analyze the behavior of edge devices to infer useful information. In this article, we exploited the vulnerabilities of current machine unlearning strategies in edge computing and proposed a new inference attack to highlight the potential privacy risk. Furthermore, we developed a defense method against this particular type of attack and proposed the price of unlearning ( PoU ) as a means to evaluate the inefficiency it brings to an edge computing system. We provide theoretical analyses to show the upper bound of the PoU using tools borrowed from game theory. The experimental results on real-world datasets demonstrate that the proposed defense strategy is effective and capable of preventing an adversary from deducing useful information.

A key management service (KMS) is vital to modern mission-critical systems. At the core of KMS are the key generation process and the key refresh process. In this paper, we design and implement a purely asynchronous system for completely distributed KMS supporting traditional applications such as threshold cryptosystems and multiparty computation (MPC) as well as emerging blockchains and Web3 applications. In this system, we have built a number of new asynchronous distributed key generation (ADKG) protocols and their corresponding asynchronous distributed key refresh (ADKR) protocols. We have demonstrated that our ADKG and ADKR protocols in the standard model outperform existing ones of the same kind, while our protocols in the random oracle model (ROM) are more efficient than other protocols with small and medium-sized networks.

Vertical Federated Learning (VFL) offers a novel paradigm in machine learning, enabling distinct entities to train models cooperatively while maintaining data privacy. This method is particularly pertinent when entities possess datasets with identical sample identifiers but diverse attributes. Recent privacy regulations emphasize an individual'sright to be forgotten, which necessitates the ability for models to unlearn specific training data. The primary challenge is to develop a mechanism to eliminate the influence of a specific client from a model without erasing all relevant data from other clients. Our research investigates the removal of a single client's contribution within the VFL framework. We introduce an innovative modification to traditional VFL by employing a mechanism that inverts the typical learning trajectory with the objective of extracting specific data contributions. This approach seeks to optimize model performance using gradient ascent, guided by a pre-defined constrained model. We also introduce a backdoor mechanism to verify the effectiveness of the unlearning procedure. Our method avoids fully accessing the initial training data and avoids storing parameter updates. Empirical evidence shows that the results align closely with those achieved by retraining from scratch. Utilizing gradient ascent, our unlearning approach addresses key challenges in VFL, laying the groundwork for future advancements in this domain.

With the growing need to comply with privacy regulations and respond to user data deletion requests, integrating machine unlearning into IoT-based federated learning has become imperative. This article introduces an innovative framework that melds blockchain with federated learning, ensuring an immutable record of unlearning requests and actions. Our approach not only bolsters the trustworthiness and integrity of the federated learning model but also adeptly addresses efficiency and security challenges typical in IoT environments. Key contributions include a certification mechanism for the unlearning process, enhancement of data security and privacy, and optimization of data management. Experimental results on MNIST and CIFAR-10 datasets demonstrate the effectiveness of our approach, achieving 0% accuracy for unlearned classes while maintaining 77.74% and 42.65% overall model accuracy for MNIST and CIFAR-10, respectively. Our time complexity analysis shows that the blockchain integration introduces only 2 seconds of overhead per epoch, highlighting the practicality of our solution for IoT applications.

Research on federated learning has continued to develop over the past few years. Many federated learning algorithms and frameworks have been developed to ensure model accuracy and protect client data privacy, which has been extensively beneficial for the development of artificial intelligence security technology. However, it is possible to recover private training data from publicly shared gradients, which is referred to as a data leakage attack. In this paper, we propose two feasible defense methods, based on gradient sparsification and pseudo-gradient, to defend against the state-of-the-art attack methods and achieve maximum protection of the private data of all federated learning participants. Both methods use cosine similarity to measure the angular difference between the gradients updated by the clients during training and the gradients sent back by the server. Taking the cosine similarity as a reference and aiming to protect clients' privacy while maintaining the accuracy of the global model, the clients can choose an appropriate strategy for disguising their uploaded gradient. Through extensive experiments, we demonstrate that both defense methods can protect users' private data while preserving the accuracy of the global model in federated learning.

With the development of various generative models, misinformation in news media becomes more deceptive and easier to create, posing a significant problem. However, existing datasets for misinformation study often have limited modalities, constrained sources, and a narrow range of topics. These limitations make it difficult to train models that can effectively combat real-world misinformation. To address this, we propose a comprehensive, large-scale Multimodal Misinformation dataset for Media Authenticity Analysis ( M 3 A ), featuring broad sources and fine-grained annotations for topics and sentiments. To curate M 3 A , we collect genuine news content from 60 renowned news outlets worldwide and generate fake samples using multiple techniques. These include altering named entities in texts, swapping modalities between samples, creating new modalities, and misrepresenting movie content as news. M 3 A contains 708K genuine news samples and over 6M fake news samples, spanning text, images, audio, and video. M 3 A provides detailed multi-class labels, crucial for various misinformation detection tasks, including out-of-context detection and deepfake detection. For each task, we offer extensive benchmarks using state-of-the-art models, aiming to enhance the development of robust misinformation detection systems. • We present M 3 A , a large-scale multimodal misinformation dataset with diverse news samples. • M 3 A includes texts, images, audio, and videos from multiple reputable news outlets. • M 3 A addresses limitations in existing datasets in misinformation generation methods and scale. • We provide multi-class annotations in M 3 A for various key tasks in misinformation detection. • We propose benchmarks for M 3 A using state-of-the-art models and out-of-distribution testing.

The rise of Deepfake technology has sparked significant concerns due to its potential for misuse and malicious manipulation of multimedia content. Various detection approaches aimed at detecting Deepfake videos have been proposed, mostly relying on the identification of spatial and temporal artifacts. However, due to the different contexts of source images and the variety of generation techniques, current Deepfake detection methods usually perform well on training datasets, and yet generalize poorly to those unseen identities in new datasets. This issue is widely known as the generalization challenge of Deepfake detection. To address this challenge, this paper proposes an advanced spatiotemporal Deepfake video detector, named M otion- e nhanced S patiotemporal T ransformer (MeST-Former). MeST-Former is based on the spatiotemporal modeling capacity of the video Swin Transformer. The spatial and temporal features are obtained from the RGB and motion images, respectively. To enhance the generalization ability of MeST-Former to unseen identities in unseen datasets, the ID-related components in the spatial and temporal features are detached. Specifically, MeST-Former adopts the newly proposed Identity-Decoupling Attention (IDC-Att) module to disentangle the ID-related and ID-unrelated components. Only the ID-unrelated components are used to construct more generalizable spatiotemporal representations. This process makes the constructed spatiotemporal features identity-agnostic and more generalizable to unseen identities. We conducted extensive experiments to evaluate the performance of the MeST-Former. Our results indicate that MeST-Former achieves accurate and generalizable Deepfake detection performance. Notably, MeST-Former also demonstrates high efficacy in detecting AI-animated talking-head videos. • RGB and motion inputs provide distinctive spatial and temporal information. • Detaching ID-related components from original embeddings can improve the generalization capabilities of the Deepfake detector. • The Swin Transformer are powerful in modeling spatiotemporal embeddings for classification. • An effectively implemented face-cropping strategy minimizes the influence of background elements.

Malicious image tampering refers to intentionally manipulating images to make them harmful to the owners or users. It has become one of the most severe challenges to image authenticity. Conventional methods for detecting tampering by identifying visual artifacts and distortions have limitations due to the rapid advancement of image manipulation techniques, which leave fewer detectable traces. To address these challenges, we propose a proactive media authentication method using deep learning-based semi-fragile watermarks. The designed scheme utilizes deep neural networks to embed an invisible watermark into a target image that is pixel-by-pixel entangled with it, which acts as an indicator of tampering trails. Once the watermarked image is counterfeited, the embedded watermark will exhibit changes accordingly, so we can locate the tampered regions by comparing retrieved and original watermarks. This proactive authentication mechanism makes our method effective against various image tamper techniques, including image copy&move, splicing and in-painting. Although our watermark is designed to be fragile to malicious tampering operations, it remains robust to benign image-processing operations such as JPEG compression, scaling, saturation, contrast adjustments, etc. This design enables our watermark to retain effectiveness when shared over the internet. Extensive experiments demonstrate that our method achieves state-of-the-art forgery detection with superior robustness, imperceptibility and security performance.

The rapid growth and escalating complexity of the Internet of Things (IoT) necessitate meticulous attention to ensure efficient and secure transactions among various autonomous components. To address this critical issue, this study proposes the integration of multiagent systems (MASs) and blockchain technology within the IoT domain. Uniquely, our approach employs smart contracts to manage exchanges between autonomous entities, thereby offering enhanced security, transparency, and reliability. The study introduces a set of innovative algorithms that regulate agent activities, such as creating blocks, sharing information, and conducting rating processes. Additionally, it provides a detailed analysis of their privacy and security aspects. Compared to traditional multiagent frameworks, empirical evidence demonstrates significant improvements in efficiency, adaptability, and scalability. This scholarly effort lays a robust foundation for further investigations into applying blockchain to enhance MASs, potentially paving the way for more sophisticated and context-specific strategies across various IoT fields.

Federated learning (FL) FL has emerged as a promising privacy-preserving machine-learning technology, enabling multiple clients to collaboratively train a global model without sharing raw data. With the increasing adoption of FL in Internet of Things (IoT) scenarios, concerns about security and privacy have become critical. In particular, gradient inversion attacks and poisoning attacks pose significant threats to the integrity and effectiveness of the global model. In response, we propose a comprehensive blockchain-based defense mechanism that effectively protects FL systems from such attacks. We develop a novel combination of techniques, including public blockchain level protection and private blockchain level protection, which work in tandem to prevent attackers from reconstructing figures using the obtained gradients. This unique combination of methods provides a robust defense against gradient inversion attacks in FL IoT scenarios. We conduct extensive experiments to validate the effectiveness of our proposed approach against gradient inversion and poisoning attacks. Our results demonstrate improved accuracy and stable convergence of training loss under poisoning attacks, indicating that our method can be applied to a wide range of FL IoT scenarios, enhancing both the security and privacy of distributed machine-learning systems.

Implementing federated learning within the Internet of Everything (IoE) framework poses substantial computational challenges, stemming from extensive client involvement, which can lead to escalated training expenses and diminished convergence rates. While many studies have investigated the combination of federated learning and blockchain networks, the integration of public and private chains to enhance federated learning performance remains largely unexplored. In this study, we introduce an innovative methodology that unifies public and private chains to mitigate clients’ computational demands while preserving data privacy and security, demonstrating compatibility within the IoE milieu and yielding favorable outcomes. To facilitate secure model migration and expedite training without incurring excessive computation costs, we delineate a blockchain-anchored model migration scheme tailored for resource-limited IoT infrastructures, establishing a private chain mechanism to incentivize companies possessing multiple devices or clients to prioritize model training. Employing blockchain technology guarantees trustworthiness in model migration, precluding the disclosure of devices’ confidential data. Overall, our innovative method provides an effective solution that improves the accuracy, privacy, and security of federated learning while reducing clients’ computational burdens within the context of the Internet of Everything (IoE).

Federated learning is a distributed learning paradigm where a global model is trained using data samples from multiple clients but without the necessity of sharing raw data samples. However, it comes with several significant challenges in system designs, data quality, and communications. Recent research highlights a significant concern related to data privacy leakage through reserve-engineering model gradients at a malicious server. Moreover, a global model cannot provide good utility performance for individual clients when the local training data is heterogeneous in terms of quantity, quality, and distribution. Hence, personalized federated learning is highly desirable in practice to tailor the trained model for local usage. In this paper, we propose PPFed, a unified federated learning framework to simultaneously address privacy preservation and personalization. The intuition of our framework is to learn part of the model gradients at the server and the rest of the gradients at the local clients. To evaluate the effectiveness of the proposed framework, we conduct extensive experiments across four image classification datasets to show that our framework yields better privacy and personalization performance compared to the existing methods. We also claim that privacy preservation and personalization are essentially two facets of deep learning models, offering a unique perspective on their intrinsic interrelation.

Resource allocation in an edge-based Internet of Things (IoT) systems can be a challenging task, especially when the system contains many devices. Hence, in recent years, scholars have devoted some attention to designing different resource allocation strategies. Among these strategies, reinforcement learning is considered to be one of the best methods for maximizing the efficiency of resource allocation schemes. In a typical reinforcement learning scheme, the edge hosts would be required to upload their local parameters to a central server. However, this process has privacy implications given some of the data processed by the edge hosts is likely to be highly sensitive. To tackle this privacy issue, we developed a concurrent joint reinforcement learning method based on local differential privacy. Our approach allows the edge host to add noise during local training to preserve privacy, and to make joint decisions with the central server to devise an optimal resource allocation strategy. Experiments show that this approach yields high performance while preserving the privacy of the edge hosts.

Adaptive traffic signal control (ATSC) attempts to alleviate traffic congestion by dynamically adjusting the timing of traffic lights in real time, and multiagent reinforcement learning is one of the ways these systems learn how and when to change signals. However, traffic congestion continues to be a problem in most highly populated cities. We know that the current research into ATSC still has much ground to cover in terms of traffic efficiency, global optimality, and convergence stability. Hence, in this article, we outline a method that provides an advising method to the multiagent traffic signal control based on relative location in real time. ATSC is regarded as a multiagent environment, in which each traffic intersection is an agent to observe the distribution of the number of vehicles (state) at the intersection to control the change of signal lights (action). In our learning framework, each agent can not only take action by its advantage actor–critic model but can also ask its neighboring agent for advice when it is not confident in its decision. The advice is generated by a real-time updated advising model, which is based on the state and relative location of neighboring agents. Because the advising model provides real-time feedback, we find that learning is more effective and convergence is more stable. Moreover, drawing on neighboring states during taking action avoids falling into a local optimality caused by only observing local states. Comparisons with similar methods show that our method brings a significant improvement in a range of evaluation criteria, such as queue lengths, vehicle speeds, and trip delays.

With breakthroughs in high-resolution image generation, applications for disentangled generative adversarial networks (GANs) have attracted much attention. At the same time, the privacy issues associated with GAN models have been raising many concerns. Membership inference attacks (MIAs), where an adversary attempts to determine whether or not a sample has been used to train the victim model, are a major risk with GANs. In prior research, scholars have shown that successful MIAs can be mounted by leveraging overfit images. However, high-resolution images make the existing MIAs fail due to their complexity. And the nature of disentangled GANs is such that the attributes are overfitting, which means that, for an MIA to be successful, it must likely be based on overfitting attributes. Furthermore, given the empirical difficulties with obtaining independent and identically distributed (IID) candidate samples, choosing the non-trivial attributes of candidate samples as the target for exploring overfitting would be a more preferable choice. Hence, in this paper, we propose a series of attribute-based MIAs that considers both black-box and white-box settings. The attacks are performed on the generator, and the inferences are derived by overfitting the non-trivial attributes. Additionally, we put forward a novel perspective on model generalization and a possible defense by evaluating the overfitting status of each individual attribute. A series of empirical evaluations in both settings demonstrate that the attacks remain stable and successful when using non-IID candidate samples. Further experiments illustrate that each attribute exhibits a distinct overfitting status. Moreover, manually generalizing highly overfitting attributes significantly reduces the risk of privacy leaks.

Model inversion attacks involve reconstructing the training data of a target model, which raises serious privacy concerns for machine learning models. However, these attacks, especially learning-based methods, are likely to suffer from low attack accuracy, i.e., low classification accuracy of these reconstructed data by machine learning classifiers. Recent studies showed an alternative strategy of model inversion attacks, GAN-based optimization, can improve the attack accuracy effectively. However, these series of GAN-based attacks reconstruct only class-representative training data for a class, whereas learning-based attacks can reconstruct diverse data for different training data in each class. Hence, in this paper, we propose a new training paradigm for a learning-based model inversion attack that can achieve higher attack accuracy in a black-box setting. First, we regularize the training process of the attack model with an added semantic loss function and, second, we inject adversarial examples into the training data to increase the diversity of the class-related parts (i.e., he essential features for classification tasks) in training data. This scheme guides the attack model to pay more attention to the class-related parts of the original data during the data reconstruction process. The experimental results show that our method greatly boosts the performance of existing learning-based model inversion attacks. Even when no extra queries to the target model are allowed, the approach can still improve the attack accuracy of reconstructed data. This new attack shows that the severity of the threat from learning-based model inversion adversaries is underestimated and more robust defenses are required.

Machine learning models are susceptible to a range of adversarial activities. These attacks are designed to either infer private information from the target model or deceive it. For instance, an attacker may attempt to discern if a given data example is from the model’s training set (membership inference attacks) or create adversarial examples to mislead the model to make incorrect predictions (adversarial example attacks). Numerous defense methods have been proposed to counter these attacks. However, these methods typically share two common limitations. Firstly, most are not designed to address label-only attacks, which is a newly emerged kind of attacks that rely solely on the hard labels predicted by the target model. Secondly, they are often developed to mitigate specific attacks rather than universally various attacks. To address these limitations, this paper proposes a novel defense method that focuses on the most challenging attacks, i.e., label-only attacks, and can handle various types of label-only attacks. The key idea is to strategically modify the target model’s predicted labels using a meta-reinforcement learning technique. This ensures that attackers receive incorrect labels while benign users continue to receive correct labels. Notably, the defender, i.e., the owner of the target model, can make effective decisions without knowledge of the attacker’s behavior. The experimental results demonstrate that our proposed method is an effective defense against a range of attacks, including label-only model stealing, label-only membership inference, label-only model inversion, and label-only adversarial example attacks.

Model stealing attacks involve creating copies of machine learning models that have similar functionalities to the original model without proper authorization. Such attacks raise significant concerns about the intellectual property of the machine learning models. Nonetheless, current defense mechanisms against such attacks tend to exhibit certain drawbacks, notably in terms of utility, and robustness. For example, watermarking-based defenses require victim models to be retrained for embedding watermarks, which can potentially impact the main task performance. Moreover, other defenses, especially fingerprinting-based methods, often rely on specific samples like adversarial examples to verify ownership of the target model. These approaches might prove less robust against adaptive attacks, such as model stealing with adversarial training. It remains unclear whether normal examples, as opposed to adversarial ones, can effectively reflect the characteristics of stolen models. To tackle these challenges, we propose a novel method that leverages a neural network as a decoder to inverse the suspicious model’s outputs. Inspired by model inversion attacks, we argue that this decoding process will unveil hidden patterns inherent in the original outputs of the suspicious model. Drawing from these decoding outcomes, we calculate specific metrics to determine the legitimacy of the suspicious models. We validate the efficacy of our defense technique against diverse model stealing attacks, specifically within the domain of classification tasks based on deep neural networks.

In recent years, there has been significant advancement in the field of model watermarking techniques. However, the protection of image-processing neural networks remains a challenge, with only a limited number of methods being developed. The objective of these techniques is to embed a watermark in the output images of the target generative network, so that the watermark signal can be detected in the output of a surrogate model obtained through model extraction attacks. This promising technique, however, has certain limits. Analysis of the frequency domain reveals that the watermark signal is mainly concealed in the high-frequency components of the output. Thus, we propose an overwriting attack that involves forging another watermark in the output of the generative network. The experimental results demonstrate the efficacy of this attack in sabotaging existing watermarking schemes for image-processing networks with an almost 100% success rate. To counter this attack, we propose an adversarial framework for the watermarking network. The framework incorporates a specially-designed adversarial training step, where the watermarking network is trained to defend against the overwriting network, thereby enhancing its robustness. Additionally, we observe an overfitting phenomenon in the existing watermarking method, which can render it ineffective. To address this issue, we modify the training process to eliminate the overfitting problem.

GAN-generated image detection now becomes the first line of defense against the malicious uses of machine-synthesized image manipulations such as deepfakes. Although some existing detectors work well in detecting clean, known GAN samples, their success is largely attributable to overfitting unstable features such as frequency artifacts, which will cause failures when facing unknown GANs or perturbation attacks. To overcome the issue, we propose a robust detection framework based on a novel multi-view image completion representation. The framework first learns various view-to-image tasks to model the diverse distributions of genuine images. Frequency-irrelevant features can be represented from the distributional discrepancies characterized by the completion models, which are stable, generalized, and robust for detecting unknown fake patterns. Then, a multi-view classification is devised with elaborated intra- and inter-view learning strategies to enhance view-specific feature representation and cross-view feature aggregation, respectively. We evaluated the generalization ability of our framework across six popular GANs at different resolutions and its robustness against a broad range of perturbation attacks. The results confirm our method's improved effectiveness, generalization, and robustness over various baselines.

In this paper, we propose the Temporal Identity Inconsistency Network (TI2Net), a Deepfake detector that focuses on temporal identity inconsistency. Specifically, TI2Net recognizes fake videos by capturing the dissimilarities of human faces among video frames of the same identity. Therefore, TI2Net is a reference-agnostic detector and can be used on unseen datasets. For a video clip of a given identity, identity information in all frames will first be encoded to identity vectors. TI2Net learns the temporal identity embedding from the temporal difference of the identity vectors. The temporal embedding, representing the identity inconsistency in the video clip, is finally used to determine the authenticity of the video clip. During training, TI2Net incorporates triplet loss to learn more discriminative temporal embeddings. We conduct comprehensive experiments to evaluate the performance of the proposed TI2Net. Experimental results indicate that TI2Net generalizes well to unseen manipulations and datasets with unseen identities. Besides, TI2Net also shows robust performance against compression and additive noise.

The explosive progress of Deepfake techniques poses unprecedented privacy and security risks to our society by creating real-looking but fake visual content. The current Deepfake detection studies are still in their infancy because they mainly rely on capturing artifacts left by a Deepfake synthesis process as detection clues, which can be easily removed by various distortions (e.g. blurring) or advanced Deepfake techniques. In this paper, we propose a novel method that does not depend on identifying the artifacts but resorts to the mechanism of anti-counterfeit labels to protect face images from malicious Deepfake tampering. Specifically, we design a neural network with an encoder-decoder structure to embed watermarks as anti-Deepfake labels into the facial identity features. The injected label is entangled with the facial identity feature, so it will be sensitive to face swap translations (i.e., Deepfake) and robust to conventional image modifications (e.g., resize and compress). Therefore, we can identify whether watermarked images have been tampered with by Deepfake methods according to the label’s existence. Experimental results demonstrate that our method can achieve average detection accuracy of more than 80%, which validates the proposed method’s effectiveness in implementing Deepfake detection.

Summary Due to successful applications of data analysis technologies in many fields, various institutions have accumulated a large amount of data to improve their services. As the speed of data collection has increased dramatically over the last few years, an increasing number of users are growing concerned about their personal information. Therefore, privacy preservation has become an urgent problem to be solved. Differential privacy as a strong privacy preservation tool has attracted significant attention. In this review, we focus on improving data utility of differentially private mechanisms through technologies related to cryptography. In particular, we first focus on how to improve data utility through anonymous communication. Then, we summarize how to improve data utility by combining differentially private mechanisms with homomorphic encryption schemes. Next, we summarize hardness results of what is impossible to achieve for differentially private mechanisms' data utility from the view of cryptography. Differential privacy borrowed intuitions from cryptography and still benefits from the progress of cryptography. To summarize the state‐of‐the‐art and to benefit future researches, we are motivated to provide this review.

Summary Privacy leakage in images attracts increasing concerns these days, as photos uploaded to large social platforms are usually not processed by proper privacy protection mechanisms. Moreover, with advanced artificial intelligence (AI) tools such as deep neural network (DNN), an adversary can detect people's identities and collect other sensitive personal information from images at an unprecedented scale. In this paper, we introduce a novel face image de‐identification framework using adversarial perturbations in the feature space. Manipulating the feature space vector ensures the good transferability of our framework. Moreover, the proposed feature space adversarial perturbation generation algorithm can successfully protect the identity‐related information while ensuring the other attributes remain similar. Finally, we conduct extensive experiments on two face image datasets to evaluate the performance of the proposed method. Our results show that the proposed method can generate real‐looking privacy‐preserving images efficiently. Although our framework has only been tested on two real‐life face image datasets, it can be easily extended to other types of images.

Abstract Collecting and analysing customers' data plays an essential role in the more intense market competition. It is critical to perform data analysis effectively while ensuring the user's privacy, especially after various privacy regulations are enacted. In this paper, we consider the problem of aggregating the stream data generated from wearable devices in a specific time period in a privacy‐preserving manner. Specifically, we adopt the local differential privacy mechanism to provide a strong privacy guarantee for users. One major challenge is that all values of the stream need to be perturbed. The additive noise makes it hard to release an accurate data stream. One way to reduce the noise scale is to select some data points to perturb instead of all. The intuition is that more privacy budgets are applied to a single data point, which ensures the statistical accuracy. The perturbed data points are used to predict the un‐selected data points without consuming an extra privacy budget. Based on this idea, we propose a novel stream data statistical framework, which includes four components, data fitting, skeleton point selection, noisy stream generation, and data aggregation. Extensive experiment results show that our proposed method achieves a much smaller mean square error given a fixed privacy budget compared with the state‐of‐the‐art.

As a result of the rapid development of Internet of Things (IoT) systems, an increasing number of academics are focusing on finding new applications for IoT systems. For IoT systems, crowdsourcing is a prevalent practise. Due to the large number of deployed devices in IoT networks, more research is still required on the privacy and trust issues that arise when utilizing crowdsourcing. As a result of the characteristics of social computing, the crowdsourcing network poses issues in terms of confidentiality and reliability. To consolidate and create this industry, we have built a differentially private crowdsourcing system that integrates public and private blockchains to address the privacy and trust issues of conventional crowdsourcing systems. Our proposed solution enables varying levels of privacy protection to protect the user’s identity and location. Moreover, the installation of blockchain networks might potentially ensure the data’s integrity. In the conclusion of this article, the possibility of deploying a crowdsourcing system with blockchain in IoE networks is examined.

Abstract Machine learning is widely deployed in society, unleashing its power in a wide range of applications owing to the advent of big data. One emerging problem faced by machine learning is the discrimination from data, and such discrimination is reflected in the eventual decisions made by the algorithms. Recent study has proved that increasing the size of training (labeled) data will promote the fairness criteria with model performance being maintained. In this work, we aim to explore a more general case where quantities of unlabeled data are provided, indeed leading to a new form of learning paradigm, namely fair semi-supervised learning. Taking the popularity of graph-based approaches in semi-supervised learning, we study this problem both on conventional label propagation method and graph neural networks, where various fairness criteria can be flexibly integrated. Our developed algorithms are proved to be non-trivial extensions to the existing supervised models with fairness constraints. Extensive experiments on real-world datasets exhibit that our methods achieve a better trade-off between classification accuracy and fairness than the compared baselines.

Internet-of-Things (IoT) are increasingly operating in the zero-trust environments where any devices and systems may be compromised and hence untrusted. In addition, data collected by and sent from IoT devices may be shared with and processed by edge computing systems, in order to reduce the reliance on centralized (cloud) servers, leading to further security and privacy issues. To cope with these challenges, this paper proposes an innovative blockchain-enabled information sharing solution in zero-trust context to guarantee anonymity yet entity authentication, data privacy yet data trustworthiness, and participant stimulation yet fairness. This new solution is able to support filtering of fabricated information through smart contracts, effective voting, and consensus mechanisms, which can prevent unauthenticated participants from sharing garbage information. We also prove that the proposed solution is secure in the universal composability framework, and further evaluate its performance over an Ethereum-based blockchain platform to demonstrate its utility.

DeepFakes are raising significant social concerns. Although various Despite various DeepFake detectors having been developed as countermeasures, their vulnerability under attacks remains further explorations. Recently, several attacks, such as adversarial attacks, have successfully fooled DeepFake detectors. However, existing attacks suffer from detector-specific designs, requiring detector-side knowledge, leading to poor transferability. Moreover, they only consider simplified security scenarios; but less is known about the attacking performance in complex scenarios where the capability of detectors or attackers varies. To fill the gap, we propose a novel, detector-agnostic trace removal attack. The attack removes all possible counterfeiting traces arising from the original DeepFake manufacture procedure to make DeepFakes essentially more "realistic" and thus able to defeat arbitrary or unknown detectors. Concretely, we first perform an in-depth DeepFake trace discovery, identifying three intrinsic traces: spatial anomalies, spectral disparities, and noise fingerprints. Then an adversarial learning-based trace removal network (TR-Net) involving one generator and multiple discriminators is proposed. Each discriminator is responsible for one individual trace representation to avoid inner-trace interference. All discriminators are optimized in parallel to enforce the generator to remove various traces simultaneously. We additionally craft heterogeneous security scenarios where the detectors are embedded with different levels of defense and the attackers own varying background data knowledge. The experimental results show that the proposed trace removal attack can significantly compromise the detection accuracy of six state-of-the-art DeepFake detectors while causing only a negligible degradation in visual quality.

In order to remain competitive, Internet companies collect and analyse user data for the purpose of the improvement of user experiences. Frequency estimation is a widely used statistical tool, which could potentially conflict with the relevant privacy regulations. Privacy preserving analytic methods based on differential privacy have been proposed, which require either a large user base or a trusted server. Although the requirements for such solutions may not be a problem for larger companies, they may be unattainable for smaller organizations. To address this issue, we propose a distributed privacy-preserving sampling-based frequency estimation method which has high accuracy even in the scenario with a small number of users while not requiring any trusted server. This is achieved by combining multi-party computation and sampling techniques. We also provide a relation between its privacy guarantee, output accuracy, and the number of participants. Distinct from most existing methods, our methods achievecentralizeddifferential privacy guarantee without the need of any trusted server. We established that, even for a small number of participants, our mechanisms can produce estimates with high accuracy and hence they provide smaller companies with more opportunity for growth through privacy-preserving statistical analysis. We further propose an architectural model to support weighted aggregation in order to achieve a higher accuracy estimate to cater for users with varying privacy requirements. Compared to the unweighted aggregation, our method provides a more accurate estimate. Extensive experiments are conducted to show the effectiveness of the proposed methods.

Recent research has discovered that deep learning models are vulnerable to membership inference attacks, which can reveal whether a sample is in the training dataset of the victim model or not. Most membership inference attacks rely on confidence scores from the victim model for the attack purpose. However, a few studies indicate that prediction labels of the victim model's output are sufficient for launching successful attacks. Besides the well-studied classification models, segmentation models are also vulnerable to this type of attack. In this article, for the first time, we propose the label-only membership inference attacks against semantic segmentation models. With a well-designed framework of the attacks, we can achieve a considerably higher successful attacking rate compared to previous work. In addition, we have discussed several possible defense mechanisms to counter such a threat.

Advanced persistent threats (APTs) are one of today’s major threats to cyber security. Highly determined attackers along with novel and evasive exfiltration techniques mean APT attacks elude most intrusion detection and prevention systems. The result has been significant losses for governments, organizations, and commercial entities. Intriguingly, despite greater efforts to defend against APTs in recent times, frequent upgrades in defense strategies are not leading to increased security and protection. In this paper, we demonstrate this phenomenon in an appropriately designed APT rivalry game that captures the interactions between attackers and defenders. What is shown is that the defender’s strategy adjustments actually leave useful information for the attackers, and thus intelligent and rational attackers can improve themselves by analyzing this information. Hence, a critical part of one’s defense strategy must be finding a suitable time to adjust one’s strategy to ensure attackers learn the least possible information. Another challenge for defenders is determining how to make the best use of one’s resources to achieve a satisfactory defense level. In support of these efforts, we figured out the optimal timings of a player’s strategy adjustment in terms of information leakage, which form a family of Nash equilibria. Moreover, two learning mechanisms are proposed to help defenders find an appropriate defense level and allocate their resources reasonably. One is based on adversarial bandits, and the other is based on deep reinforcement learning. Experimental simulations show the rationales behind the game and the optimality of the equilibria. The results also demonstrate that players indeed have the ability to improve themselves by learning from past experiences, which shows the necessity of specifying optimal strategy adjustment timings when defending against APTs.

Over the past decades, the abundance of personal data has led to the rapid development of machine learning models and important advances in artificial intelligence (AI). However, alongside all the achievements, there are increasing privacy threats and security risks that may cause significant losses for data providers. Recent legislation requires that the private information about a user should be removed from a database as well as machine learning models upon certain deletion requests. While erasing data records from memory storage is straightforward, it is often challenging to remove the influence of particular data samples from a model that has already been trained. Machine unlearning is an emerging paradigm that aims to make machine learning models “forget” what they have learned about particular data. Nevertheless, the unlearning issue for federated learning has not been completely addressed due to its special working mode. First, existing solutions crucially rely on retraining-based model calibration, which is likely unavailable and can pose new privacy risks for federated learning frameworks. Second, today’s efficient unlearning strategies are mainly designed for convex problems, which are incapable of handling more complicated learning tasks like neural networks. To overcome these limitations, we took advantage of differential privacy and developed an efficient machine unlearning algorithm named FedRecovery. The FedRecovery erases the impact of a client by removing a weighted sum of gradient residuals from the global model, and tailors the Gaussian noise to make the unlearned model and retrained model statistically indistinguishable. Furthermore, the algorithm neither requires retraining-based fine-tuning nor needs the assumption of convexity. Theoretical analyses show the rigorous indistinguishability guarantee. Additionally, the experiment results on real-world datasets demonstrate that the FedRecovery is efficient and is able to produce a model that performs similarly to the retrained one.

In a model inversion attack, an adversary attempts to reconstruct the training data records of a target model using only the model’s output. In launching a contemporary model inversion attack, the strategies discussed are generally based on either predicted confidence score vectors, i.e., black-box attacks, or the parameters of a target model, i.e., white-box attacks. However, in the real world, model owners usually only give out the predicted labels; the confidence score vectors and model parameters are hidden as a defense mechanism to prevent such attacks. Unfortunately, we have found a model inversion method that can reconstruct representative samples of the target model’s training data based only on the output labels. We believe this attack requires the least information to succeed and, therefore, has the best applicability. The key idea is to exploit the error rate of the target model to compute the median distance from a set of data records to the decision boundary of the target model. The distance is then used to generate confidence score vectors which are adopted to train an attack model to reconstruct the representative samples. The experimental results show that highly recognizable representative samples can be reconstructed with far less information than existing methods.

With the ubiquity of social media, privacy leakage has become a urgent problemfor social media managers. Studying how the privacy information diffuses through social media has attracted much attention. As a prerequisite, modeling privacy information diffusion is important research. Current approaches for modeling information diffusion are not available for privacy information since they did not consider the propagation features of privacy information in social media. Thispaper discusses the problem of modeling privacy information in social media and its challenges. We first analyse the information diffusion paths in the basic parameters of complex network and the high-order structures. We find that the privacy information is different in propagation features and the size of star structures. Second, a new information diffusion model is illustrated to simulate the diffusion process of information in social media by considering the following three parameters: 1) the probability of users receiving this message, 2) the probability that users have a tendency to forward this message and 3) the interest the users hold for this message. Finally, a block mechanism is designed to congest the diffusion of privacy information in social media.

Social media has become a ubiquitous tool for spreading news, messages, and generally allowing for communication between individuals. Hence, studying how our privacy information might also spread across social media is important research. To date, many studies have used information diffusion models to simulate and then examine how information flows through social networks. But these models are theoretical, and newsworthy information may not behave in the same way as privacy information, raising the question: Are the observed phenomena indicative of real privacy propagation? To explore this question, we assembled a dataset from Twitter comprising propagated information flows for both private and normal information. We then built a graph convolutional network to trace and classify differences in the way each type of information spreads throughout the platform. The results reveal that there are indeed key differences in the diffusion processes of the two types of information. More importantly, we design privacy-preserving methods to reduce the privacy propagation in social media.

KNN has gained popularity in machine learning due to its simplicity and good performance. However, kNN faces two problems with classification tasks. The first is that an appropriate distance measurement is required to compute distances between test sample and training samples. The other is the highly computational complexity due to the requirement of searching the nearest neighbors in the whole training data. In order to mitigate these two problems, we propose a novel method named KCNN to enhance the performance of kNN. KCNN uses convolutional neural networks to learn a suitable distance metric as well as prototype reduction to learn a reduced set of prototypes which can represent the original set. It has several superiorities compared with related methods. The combination of CNN and kNN empowers it to extract discriminative hierarchical features with which kNN can easily classify. KCNN learns spatial information on an image instead of considering it as a vector to learn distance metric. Moreover, KCNN simultaneously learns a reduced set of prototypes, which help improve classification efficiency and avoid noisy samples of the massive training set. The proposed method has a better robustness and convergence than CNN, especially when projecting input data into a low-dimension space.

Deep generative models have gained much attention given their ability to generate data for applications as varied as healthcare to financial technology to surveillance, and many more - the most popular models being generative adversarial networks (GANs) and variational auto-encoders (VAEs). Yet, as with all machine learning models, ever is the concern over security breaches and privacy leaks and deep generative models are no exception. In fact, these models have advanced so rapidly in recent years that work on their security is still in its infancy. In an attempt to audit the current and future threats against these models, and to provide a roadmap for defense preparations in the short term, we prepared this comprehensive and specialized survey on the security and privacy preservation of GANs and VAEs. Our focus is on the inner connection between attacks and model architectures and, more specifically, on five components of deep generative models: the training data, the latent code, the generators/decoders of GANs/VAEs, the discriminators/encoders of GANs/VAEs, and the generated data. For each model, component and attack, we review the current research progress and identify the key challenges. The paper concludes with a discussion of possible future attacks and research directions in the field.

Federated learning is a promising distributed machine learning paradigm that has been playing a significant role in providing privacy-preserving learning solutions. However, alongside all its achievements, there are also limitations. First, traditional frameworks assume that all the clients are voluntary and so will want to participate in training only for improving the model’s accuracy. However, in reality, clients usually want to be adequately compensated for the data and resources they will use before participating. Second, today’s frameworks do not offer sufficient protection against malicious participants who try to skew a jointly trained model with poisoned updates. To address these concerns, we have developed a more robust federated learning scheme based on joint differential privacy. The framework provides two game-theoretic mechanisms to motivate clients to participate in training. These mechanisms are dominant-strategy truthful, individual rational, and budget-balanced. Further, the influence an adversarial client can have is quantified and restricted, and data privacy is similarly guaranteed in quantitative terms. Experiments with different training models on real-word datasets demonstrate the effectiveness of the proposed approach.

Federated learning is a promising distributed machine learning paradigm that has been playing a significant role in privacy-preserving machine learning tasks. However, alongside all its achievements, the framework has limitations. First, traditional frameworks assume that all clients want to improve model accuracy and so participation is voluntary. However, in reality, clients usually want to be appropriately compensated for the data and resources they will need to commit to the training process before contributing. Second, today's frameworks allow clients to perturb their parameter updates locally, which introduces a great deal of noise to the trained model and can seriously impact model accuracy. To address these concerns, we have developed a private reward game that incentivizes clients to contribute high-quality data to the training process. The game converges to a Nash equilibrium under the guarantee of joint differential privacy, and each client maximizes their reward following an equilibrium strategy. The noise injected into the model is reduced by introducing a centralized differential privacy model that aggregates the parameters and compensates clients via a data trading market. Experimental simulations show the rationales behind and effectiveness of the proposed game approach. Additionally, we present comparisons between different training models to demonstrate the performance of the proposed approach in real-world scenarios.

In multiagent learning, one of the main ways to improve learning performance is to ask for advice from another agent. Contemporary advising methods share a common limitation that a teacher agent can only advise a student agent if the teacher has experience with an identical state. However, in highly complex learning scenarios, such as autonomous driving, it is rare for two agents to experience exactly the same state, which makes the advice less of a learning aid and more of a one-time instruction. In these scenarios, with contemporary methods, agents do not really help each other learn, and the main outcome of their back and forth requests for advice is an exorbitant communications' overhead. In human interactions, teachers are often asked for advice on what to do in situations that students are personally unfamiliar with. In these, we generally draw from similar experiences to formulate advice. This inspired us to provide agents with the same ability when asked for advice on an unfamiliar state. Hence, we propose a model-based self-advising method that allows agents to train a model based on states similar to the state in question to inform its response. As a result, the advice given can not only be used to resolve the current dilemma but also many other similar situations that the student may come across in the future via self-advising. Compared with contemporary methods, our method brings a significant improvement in learning performance with much lower communication overheads.

As deep learning models mature, one of the most prescient questions we face is: what is the ideal tradeoff between accuracy, fairness, and privacy (AFP)? Unfortunately, both the privacy and the fairness of a model come at the cost of its accuracy. Hence, an efficient and effective means of fine-tuning the balance between this trinity of needs is critical. Motivated by some curious observations in privacy-accuracy tradeoffs with differentially private stochastic gradient descent (DP-SGD), where fair models sometimes result, we conjecture that fairness might be better managed as an indirect byproduct of this process. Hence, we conduct a series of analyses, both theoretical and empirical, on the impacts of implementing DP-SGD in deep neural network models through gradient clipping and noise addition. The results show that, in deep learning, the number of training epochs is central to striking a balance between AFP because DP-SGD makes the training less stable, providing the possibility of model updates at a low discrimination level without much loss in accuracy. Based on this observation, we designed two different early stopping criteria to help analysts choose the optimal epoch at which to stop training a model so as to achieve their ideal tradeoff. Extensive experiments show that our methods can achieve an ideal balance between AFP.

The volume, variety and value of data generated by Internet of Things (IoT) devices are expected to increase significantly in foreseeable future, hence, reinforcing the importance of secure and efficient access control solutions for these devices and their networks. However, existing access control solutions are not generally lightweight or scalable, particularly for geographically disperse, inexpensive resource constrained IoT devices. To tackle above challenges, we propose a lightweight consortium blockchain based architecture to enable intelligent autonomous access control for IoT devices. In our architecture, intelligent blockchain facilitates the storing of access policies, provision of authentication services for data access control, and trust evaluation for access request nodes through token accumulation mechanism. Specifically, the user's access request is approved only after it is confirmed by the blockchain network. To ensure the reliability of authenticity, a compromised resistant consensus algorithm is adapted and implemented to defend against at most$1/3$compromised authenticators. In addition, a cross-domain and flexible access control model is not only used to support data sharing among various users but can also be used for access control for exceptional blockchain situations. We explain how our system meets our design goals of reliability, availability, confidentiality, integrity, lightweight, security and scalability. In addition, we also analyze the proposed system's performance from computational, storage and network overheads (e.g., running cryptographic algorithms on a Raspberry Pi 4B), and the findings suggest that the time to run typical cryptographic algorithms is in the microsecond range.

A cyber system may face to multiple attackers from diverse adversaries, who usually employ sophisticated techniques to both continuously steal sensitive data and avoid being detected by defense strategies. This continuous process is typically involved in an advanced persistent threat (APT). Since the game theory is an ideal mathematical model for investigating continuous decision making of competing players, it is broadly used to research the interaction between defenders and APT attackers. Although many researchers are now using the game theory to defend against APT attacks, most of the existing solutions are limited to single-defender, single-attacker scenarios. In the real world, threats by multiple attackers are not uncommon and multiple defenders can be put in place. Therefore, to overcome the limitation of the existing solutions, we develop a multiagent deep reinforcement learning (MADRL) method with a novel sampling approach. The MADRL method allows defenders to create strategies on the fly and share their experience with other defenders. To develop this method, we create a multidefender, multiattacker game model and analyze the equilibrium of this model. The results of a series of experiments demonstrate that, with MADRL, defenders can quickly learn efficient strategies against attackers.

Summary Differential privacy is a rigorous mathematical framework for evaluating and protecting data privacy. In most existing studies, there is a vulnerable assumption that records in a dataset are independent when differential privacy is applied. However, in real‐world datasets, records are likely to be correlated, which may lead to unexpected data leakage. In this survey, we investigate the issue of privacy loss due to data correlation under differential privacy models. Roughly, we classify existing literature into three lines: (1) using parameters to describe data correlation in differential privacy, (2) using models to describe data correlation in differential privacy, and (3) describing data correlation based on the framework of Pufferfish. First, a detailed example is given to illustrate the issue of privacy leakage on correlated data in real scenes. Then our main work is to analyze and compare these methods, and evaluate situations that these diverse studies are applied. Finally, we propose some future challenges on correlated differential privacy.

Summary Federated learning is an improved version of distributed machine learning that further offloads operations which would usually be performed by a central server. The server becomes more like an assistant coordinating clients to work together rather than micromanaging the workforce as in traditional DML. One of the greatest advantages of federated learning is the additional privacy and security guarantees it affords. Federated learning architecture relies on smart devices, such as smartphones and IoT sensors, that collect and process their own data, so sensitive information never has to leave the client device. Rather, clients train a submodel locally and send an encrypted update to the central server for aggregation into the global model. These strong privacy guarantees make federated learning an attractive choice in a world where data breaches and information theft are common and serious threats. This survey outlines the landscape and latest developments in data privacy and security for federated learning. We identify the different mechanisms used to provide privacy and security, such as differential privacy, secure multiparty computation and secure aggregation. We also survey the current attack models, identifying the areas of vulnerability and the strategies adversaries use to penetrate federated systems. The survey concludes with a discussion on the open challenges and potential directions of future work in this increasingly popular learning paradigm.

Summary Due to the rapid development of the cloud computing environment, it is widely accepted that cloud servers are important for users to improve work efficiency. Users need to know servers' capabilities and make optimal decisions on selecting the best available servers for users' tasks. We consider the process of learning servers' capabilities by users as a multiagent reinforcement learning process. The learning speed and efficiency in reinforcement learning can be improved by sharing the learning experience among learning agents which is defined as advising. However, existing advising frameworks are limited by the requirement that during advising all learning agents in a reinforcement learning environment must have exactly the same actions. To address the above limitation, this article proposes a novel differentially private advising framework for multiagent reinforcement learning. Our proposed approach can significantly improve the application of conventional advising frameworks when agents have one different action. The approach can also widen the applicable field of advising and speed up reinforcement learning by triggering more potential advising processes among agents with different actions.

Summary Privacy risk assessment plays a fundamental role in privacy preservation, as it determines the extent to which subsequent processing (such as generalization and obfuscation), should be applied to the sensitive data. However, most existing works on privacy risk assessment have focused on structured data, while unstructured text data remain relatively underexplored due to the complexity of natural language. In this article, we propose a novel method, PriTxt, for evaluating the privacy risk associated with text data by exploiting the semantic correlation. Using definitions derived from the General Data Protection Regulation (GDPR), a de facto standard of privacy preservation in practice, PriTxt first defines the private features that related to individual privacy in order to locate the sensitive words. By using the word2vec algorithm, a word‐embedding model is further constructed to identify the quasi‐sensitive words that are semantically correlated to the private features. The privacy risk of a given text is finally evaluated by aggregating the weighted risks of the sensitive and the quasi‐sensitive words in the text. Experiments on real‐world datasets demonstrate that the proposed PriTxt is effective for conducting risk assessment on text data and further outperforms the traditional methods.

Summary The explosive growth of various computer vision technologies generates a tremendous amount of visual data online every day. In addition to bringing convenience and revolutionizing our daily life, image data also reveal a wide range of sensitive information and pose unprecedented privacy leakage risks. Particularly, in the case of photos contain human faces, people can easily access those face images on social media without any consent, and the misuse of personal information could cause serious privacy violation to individuals. Therefore, it is essential to consider sanitizing people's identity information when using images containing human faces. As a result, there has been rapid development in the area of facial anonymization, also called image de‐identification. However, due to the emergence of numerous deep‐learning based attacks, traditional anonymization methods such as blurring and mosaic are weak and ineffective to protect individual's privacy in face images. To respond to this challenge, this article proposes a novel de‐identification method that utilizes a deep neural network. The proposed framework encompasses two modules: encoder network and generator network. The encoder transforms a face image into a high‐semantic latent vector of codes, which will be de‐identified according to the differential privacy criterion. The generator leverages the unconditional generative adversarial network to synthesize high‐quality images based on the modified latent codes from the encoder. Extensive experimental results indicate that our proposed model can protect image privacy while keeping the processed image visual realistic.

In multi-agent reinforcement learning, transfer learning is one of the key techniques used to speed up learning performance through the exchange of knowledge among agents. However, there are three challenges associated with applying this technique to real-world problems. First, most real-world domains are partially rather than fully observable. Second, it is difficult to pre-collect knowledge in unknown domains. Third, negative transfer impedes the learning progress. We observe that differentially private mechanisms can overcome these challenges due to their randomization property. Therefore, we propose a novel differential transfer learning method for multi-agent reinforcement learning problems, characterized by the following three key features. First, our method allows agents to implement real-time knowledge transfers between each other in partially observable domains. Second, our method eliminates the constraints on the relevance of transferred knowledge, which expands the knowledge set to a large extent. Third, our method improves robustness to negative transfers by applying differentially exponential noise and relevance weights to transferred knowledge. The proposed method is the first to use the randomization property of differential privacy to stimulate the learning performance in multi-agent reinforcement learning system. We further implement extensive experiments to demonstrate the effectiveness of our proposed method.

Massive Internet of Things (IoT) data sets are possessed by big institutions serving daily life because IoT devices are widely used in our daily life such as wearable devices and smart home devices. Publishing these data sets among various institutions causes an increasing number of users to concern their personal privacy. Differential privacy is the state-of-the-art concept of privacy preservation, but it suffers from the low accuracy. In this article, we improve differentially private mechanisms including the Laplace mechanism as well as the sample and aggregation mechanism by bringing the personalized sampling technology into these mechanisms so that IoT data sets can be privately published through differentially private mechanisms. In particular, improved mechanisms assign a personalized sampling probability to each data record in a way that their accuracy can be improved. We analyse improved mechanisms in terms of their privacy and accuracy. Then, we empirically demonstrate that the performance of improved mechanisms is better than original mechanisms through extensive experiments on synthetic data sets and real-world data sets.

Resource allocation is a fundamental research issue in IoT edge computing, and reinforcement learning is fast becoming a common solution. The majority of the current techniques involve decision makers who determine how and where resources should be distributed. In a standard cloud system, this decision maker is a central server. In an edge system, the decision maker is an edge host. Both approaches have drawbacks. Edge hosts do not always have access to enough global information to create the most optimal resource allocation strategy. Central servers do but at the cost of privacy. A solution is needed that can do both. This article, therefore, presents a novel resource allocation method called concurrent federated reinforcement learning. The scheme inherits the privacy protection of federated learning, the complex problem solving power of reinforcement learning and adds concurrency in the form of joint decision making so the resource allocation strategies work to the benefit of the global system. The experiments demonstrate that the approach provides the state-of-the-art performance in system-wide utility, speed of task completion, and resource utilization.

Abstract Deep-learning based natural language processing (NLP) models are proven vulnerable to adversarial attacks. However, there is currently insufficient research that studies attacks to neural machine translations (NMTs) and examines the robustness of deep-learning based NMTs. In this paper, we aim to fill this critical research gap. When generating word-level adversarial examples in NLP attacks, there is a conventional trade-off in existing methods between the attacking performance and the amount of perturbations. Although some literature has studied such a trade-off and successfully generated adversarial examples with a reasonable amount of perturbations, it is still challenging to generate highly successful translation attacks while concealing the changes to the texts. To this end, we propose a novel Hybrid Attentive Attack method to locate language-specific and sequence-focused words, and make semantic-aware substitutions to attack NMTs. We evaluate the effectiveness of our attack strategy by attacking three high-performing translation models. The experimental results show that our method achieves the highest attacking performance compared with other existing attacking strategies.

Abstract In recent years, it has been revealed that machine learning models can produce discriminatory predictions. Hence, fairness protection has come to play a pivotal role in machine learning. In the past, most studies on fairness protection have used traditional machine learning methods to enforce fairness. However, these studies focus on low dimensional inputs, such as numerical inputs, whereas more recent deep learning technologies have encouraged fairness protection with image inputs through deep model methods. These approaches involve various object functions and structural designs that break the spurious correlations between targets and sensitive features. With these connections broken, we are left with fairer predictions. To better understand the proposed methods and encourage further development in the field, this paper summarizes fairness protection methods in terms of three aspects: the problem settings, the models, and the challenges. Through this survey, we hope to reveal research trends in the field, discover the fundamentals of enforcing fairness, and summarize the main challenges to producing fairer models.

The electronic reporting system can alleviate the problems in terms of efficiency, content confidentiality, and reporter privacy imposed in the traditional reporting system. Relying on anonymity, the privacy of reporters can be protected, but the authentication of reporters with fake names should also be maintained. If authenticated anonymity is guaranteed, the reporters may still conduct misbehaviors such as submitting fake reports after the authentication. To address the above dilemma, we propose to apply a proxy signature to achieve authenticated anonymity and employ blockchain to maintain anonymity yet guarantee traceability for reporters’ misbehaviors. We also propose a new proxy signature scheme in this paper by module lattice for postquantum security. The extensive analysis justified our proposed scheme is secure and manageable.

Agent advising is one of the main approaches to improve agent learning performance by enabling agents to share advice. Existing advising methods have a common limitation that an adviser agent can offer advice to an advisee agent only if the advice is created in the same state as the advisee's state. However, in complex environments, it is a very strong requirement that two states are the same, because a state may consist of multiple dimensions and two states being the same means that all these dimensions in the two states are correspondingly identical. Therefore, this requirement may limit the applicability of existing advising methods to complex environments. In this article, inspired by the differential privacy scheme, we propose a differential advising method that relaxes this requirement by enabling agents to use advice in a state even if the advice is created in a slightly different state. Compared with the existing methods, agents using the proposed method have more opportunity to take advice from others. This article is the first to adopt the concept of differential privacy on advising to improve agent learning performance instead of addressing security issues. The experimental results demonstrate that the proposed method is more efficient in complex environments than the existing methods.

Planning is one of the main approaches used to improve agents’ working efficiency by making plans beforehand. However, during planning, agents face the risk of having their private information leaked. This article proposes a novel strong privacy-preserving planning approach for logistic-like problems. This approach outperforms existing approaches by addressing two challenges: 1) simultaneously achieving strong privacy; completeness and efficiency; and 2) addressing communication constraints. These two challenges are prevalent in many real-world applications including logistics in military environments and packet routing in networks. To tackle these two challenges, our approach adopts the differential privacy technique, which can both guarantee strong privacy and control communication overhead. To the best of our knowledge, this article is the first to apply differential privacy to the field of multi-agent planning as a means of preserving the privacy of agents for logistic-like problems. We theoretically prove the strong privacy and completeness of our approach and empirically demonstrate its efficiency. We also theoretically analyze the communication overhead of our approach and illustrate how differential privacy can be used to control it.

Machine learning models are vulnerable to data inference attacks, such as membership inference and model inversion attacks. In these types of breaches, an adversary attempts to infer a data record’s membership in a dataset or even reconstruct this data record using a confidence score vector predicted by the target model. However, most existing defense methods only protect against membership inference attacks. Methods that can combat both types of attacks require a new model to be trained, which may not be time-efficient. In this paper, we propose a differentially private defense method that handles both types of attacks in a time-efficient manner by tuning only one parameter, the privacy budget. The central idea is to modify and normalize the confidence score vectors with a differential privacy mechanism which preserves privacy and obscures membership and reconstructed data. Moreover, this method can guarantee the order of scores in the vector to avoid any loss in classification accuracy. The experimental results show the method to be an effective and timely defense against both membership inference and model inversion attacks with no reduction in accuracy.

A growing specter in the rise of machine learning is whether the decisions made by machine learning models are fair. While research is already underway to formalize a machine-learning concept of fairness and to design frameworks for building fair models with sacrifice in accuracy, most are geared toward either supervised or unsupervised learning. Yet two observations inspired us to wonder whether semi-supervised learning might be useful to solve discrimination problems. First, previous study showed that increasing the size of the training set may lead to a better trade-off between fairness and accuracy. Second, the most powerful models today require an enormous of data to train which, in practical terms, is likely possible from a combination of labeled and unlabeled data. Hence, in this paper, we present a framework of fair semi-supervised learning in the pre-processing phase, including pseudo labeling to predict labels for unlabeled data, a re-sampling method to obtain multiple fair datasets and lastly, ensemble learning to improve accuracy and decrease discrimination. A theoretical decomposition analysis of bias, variance and noise highlights the different sources of discrimination and the impact they have on fairness in semi-supervised learning. A set of experiments on real-world and synthetic datasets show that our method is able to use unlabeled data to achieve a better trade-off between accuracy and discrimination.

Networks that constantly transmit information and change structure are becoming increasingly prevalent. However, traditional privacy models are designed to protect static information, such as records in a database or a person’s profile information, which seldom changes. This conflict between static models and dynamic environments is dramatically hindering the effectiveness and efficiency of privacy preservation in today’s dynamic world. Hence, in this paper, we formally define the concept of dynamic privacy, present two novel perspectives, privacy propagation and accumulation, on the way private information can spread through dynamic cyberspace, and develop associated theories and mechanisms for preserving privacy in advanced complex networks, such as social networking sites where data are constantly being released, shared, and exchanged.

Artificial Intelligence (AI) has attracted a great deal of attention in recent years. However, alongside all its advancements, problems have also emerged, such as privacy violations, security issues and model fairness. Differential privacy, as a promising mathematical model, has several attractive properties that can help solve these problems, making it quite a valuable tool. For this reason, differential privacy has been broadly applied in AI but to date, no study has documented which differential privacy mechanisms can or have been leveraged to overcome its issues or the properties that make this possible. In this paper, we show that differential privacy can do more than just preserve privacy. It can also be used to improve security, stabilize learning, build fair models, and impose composition in selected areas of AI. With a focus on regular machine learning, distributed machine learning, deep learning, and multi-agent systems, the purpose of this article is to deliver a new view on many possibilities for improving AI performance with differential privacy techniques.

Abstract To alleviate the traffic congestion caused by the sharp increase in the number of private cars and save commuting costs, taxi carpooling service has become the choice of many people. Current research on taxi carpooling services has focused on shortening the detour distances. While with the development of intelligent cities, efficiently match passengers and vehicles and planning routes become urgent. And the privacy between passengers in the taxi carpooling service also needs to be considered. In this paper, we propose a time-optimal and privacy-preserving carpool route planning system via deep reinforcement learning. This system uses the traffic information around the carpooling vehicle to optimize passengers’ travel time, not only to efficiently match passengers and vehicles but also to generate detailed route planning for carpooling vehicles. We conducted experiments on an Internet of Vehicles simulator CARLA, and the results demonstrate that our method is better than other advanced methods and has better performance in complex environments.

Internet of Things (IoT) current present two trends with respect to ubiquity and mobility. The number of devices increases remarkably and most forthcoming devices are mobile, e.g., Internet of Vehicles (IoV). In large scale mobile IoT, the management of device identification, fast authentication, and certification of public keys experiences upcoming difficulties. Centralized security architecture may not be suitable and in contrast cross-domain security architecture will tackle the ubiquity and mobility better. We note that blockchain is a new decentralized architecture equipped with basic cryptographic settings, which provides new tools such as token and guarantees many security properties such as block integrity. In this paper we design a blockchain-based fast authentication video data forwarding scheme for IoV. Compared with existing protocols, it has the advantages of decentralization, high authentication efficiency, strong trust, and resistance to common attacks. We evaluate our scheme with real experiments over Hyperledger Fabric and the authentication delay is manageable (e.g, 4 seconds in IoV).

Summary Existing privacy‐preserving approaches are generally designed to provide privacy guarantee for individual data in a database, which reduces the utility of the database for data analysis. In this paper, we propose a novel differential privacy mechanism to preserve the heterogeneous privacy of a vertically partitioned database based on attributes. We first present the concept of privacy label, which characterizes the privacy information of the database and is instantiated by the classification. Then, we use an information‐based method to systematically explore the dependencies between all attributes and the privacy label. We finally assign privacy weights to every attribute and design a heterogeneous mechanism according to the basic Laplace mechanism. Evaluations using real datasets demonstrate that the proposed mechanism achieves a balanced privacy and utility.

The importance of protecting personal information, like, a person's address or health history, is well known and commonly discussed. However, images also contain sensitive information that can compromise a person's privacy or be used for nefarious purposes. To date, most methods for preserving privacy with images have relied on obfuscation techniques, such as pixelation, blurring, or masking parts of the image. However, new face-recognition technologies driven by deep learning are showing cracks in the old techniques. Moreover, faceless recognition is presenting a whole new set of challenges for image privacy. The core of these issues it is how to ensure privacy while still being able to see and use the image. Our solution is a model based on a generative adversarial network that protects identity information while preserving face features of the original image as much as possible. The premise is to generate a fake image of a face that shares all the same attributes as the original image, for example, a brown-eyed child smiling. With this strategy, the image remains useful, but no person or algorithm could determine the identity of the pictured individual. The framework consists of three parts: a detection module, an image creation module, and an image transformation module. The detection module extracts the attribute labels. The image creation module generates images of faces, and the image transformation module transforms the fake features to match the attributes in the original image. A comprehensive set of experiments shows the effectiveness of the proposed framework.

Bitcoin transactions are not truly anonymous as an attacker can attempt to reveal a user's private information by tracing related transactions. Existing approaches to protect privacy (e.g., mixcoin, shuffle, and blinded token) suffer from a number of limitations. For example, some approaches assume the existence of a trusted third party, rely on exchanges among various currencies, or broadcast sensitive details before mixing. Therefore, there is a real risk of privacy breach or losing tokens. Thus in this paper, we design a mixing scheme with one decentralized signature protocol, which does not rely on a third party or require a transaction fee. Specifically, our scheme uses a negotiation process to guarantee transaction details, which is monitored by the participants. Furthermore, the scheme includes a signature protocol based on the ElGamal signature protocol and secret sharing. The proposed scheme is then proven secure.

Cyber deception is one of the key approaches used to mislead attackers by hiding or providing inaccurate system information. There are two main factors limiting the real-world application of existing cyber deception approaches. The first limitation is that the number of systems in a network is assumed to be fixed. However, in the real world, the number of systems may be dynamically changed. The second limitation is that attackers' strategies are simplified in the literature. However, in the real world, attackers may be more powerful than theory suggests. To overcome these two limitations, we propose a novel differentially private game theoretic approach to cyber deception. In this proposed approach, a defender adopts differential privacy mechanisms to strategically change the number of systems and obfuscate the configurations of systems, while an attacker adopts a Bayesian inference approach to infer the real configurations of systems. By using the differential privacy technique, the proposed approach can 1) reduce the impacts on network security resulting from changes in the number of systems and 2) resist attacks regardless of attackers' reasoning power. The experimental results demonstrate the effectiveness of the proposed approach.

A critical challenge in machine learning is the vulnerability of learning models in defending attacks from malicious adversaries. In this research, we propose game theoretical learning between a variational adversary and a Convolutional Neural Network (CNN), participating in a variable-sum two-player sequential Stackelberg game. Our adversary manipulates the input data distribution to make the CNN misclassify the manipulated data. Our ideal adversarial manipulation is a minimum change to the data which yet is large enough to mislead the CNNs. We propose an optimization procedure to find optimal adversarial manipulations by solving for the Nash equilibrium of the Stackelberg game. Specifically, the adversary's payoff function depends on the data manipulation which is determined by a Variational Autoencoder, while the CNN classifier's payoff functions are evaluated by misclassification errors. The optimization of our adversarial manipulations is defined by Alternating Least Squares and Simulated Annealing. Experimental results demonstrate that our game-theoretic manipulations are able to mislead CNNs that are well trained on the original data as well as on data generated by other models. We then let the CNNs to incorporate our manipulated data which leads to secure classifiers that are empirically the most robust in defending various types of adversarial attacks.

Worker recruitment in mobile crowdsensing systems aims to recruit the most suitable users to perform tasks with high quality and in real-time. Many worker recruitment or task matching mechanisms have been proposed, especially for crowdsourcing platforms, where content information of tasks from the implicit feedback of workers' attendance is extensively exploited to help workers find preferred tasks efficiently. Different from traditional crowdsourcing systems, tasks in mobile crowdsensing systems are usually time-sensitive and location-dependent which also play a crucial role in worker recruitment. However, these context information have not been effectively explored for user recruitment in mobile crowdsensing systems. In this paper, we propose a novel personalized task-oriented worker recruitment mechanism for mobile crowdsensing systems based on a careful characterization of workers' preference. In particular, we fully exploit the content information (e.g., task category, task description) together with the context information (e.g., task time, task location) from the implicit feedback of workers' attendance to accurately model workers' preference on tasks. Moreover, we regard the task-worker fitness prediction as a binary classification problem and utilize the Logit model to integrate the heterogeneous factors into a single framework to predict the matching probability of each task-worker pair. Finally, the workers with the highest matching probability are recruited proactively for each new task. Extensive experiments on real-world datasets demonstrate that the proposed mechanism achieves better performance than the benchmarks.

Android piggybacked malware (i.e., apps that piggyback malicious code) are becoming ubiquitous in app stores. Malware writers often use obfuscation techniques to obfuscate piggybacked apps to evade detection by Android malware detectors. Previous studies in this field have focused on the impact of code obfuscations on the detection of piggybacked malware, but the impact of code deobfuscation on detecting obfuscated piggybacked apps has rarely been studied. Knowing about the impact of code deobfuscation can provide useful insights into obfuscated piggybacked apps and therefore the design of resilient Android malware detectors. In this paper we conduct an empirical study of code deobfuscations on detecting obfuscated Android piggybacked apps, focusing on three types of malware detectors: commercial anti-malware products, machine learning-based detectors, and similarity-based detectors. We observe that code deobfuscations can impact differently depending on the malware detectors. For example, some deobfuscation strategies can improve the precision of detecting obfuscated piggybacked apps. Also we observe that the examined deobfuscation tools (Simplify and Deguard) have a different impact on obfuscated piggybacked apps after deobfuscations.

Privacy protection attracts increasing concerns these days. People tend to believe that large social platforms will comply with the agreement to protect their privacy. However, photos uploaded by people are usually not treated to achieve privacy protection. For example, Facebook, the world's largest social platform, was found leaking photos of millions of users to commercial organizations for big data analytics. A common analytical tool used by these commercial organizations is the Deep Neural Network (DNN). Today's DNN can accurately identify people's appearance, body shape, hobbies and even more sensitive personal information, such as addresses, phone numbers, emails, bank cards and so on. To enable people to enjoy sharing photos without worrying about their privacy, we propose an algorithm that allows users to selectively protect their privacy while preserving the contextual information contained in images. The results show that the proposed algorithm can select and perturb private objects to be protected among multiple optional objects so that the DNN can only identify non-private objects in images.

The application of blockchain has moved beyond cryptocurrencies, to applications such as credentialing and smart contracts. The smart contract allows ones to achieve fair exchange for values without relying on a centralized entity. However, as the smart contract can be automatically executed with token transfers, an attacker can seek to exploit vulnerabilities in smart contracts for illicit profits. Thus, this paper proposes a support vector machine (SVM)-based scanning system for vulnerabilities on smart contracts. Our evaluation on Ethereum demonstrate that we achieve a identification rate of over 90% based on several popular attacks.

The differential privacy is the state-of-the-art conception for privacy preservation due to its strong privacy guarantees, however it suffers from low accuracy. In this paper, we propose a personalized sample Laplace mechanism by combining the Laplace mechanism with sampling technology. In order to improve the accuracy, the proposed mechanism assigns personalized sampling probability to each record. Based on the personalized sampling probability, we prove that the proposed mechanism satisfies ε differential privacy. Then we compare the proposed mechanism with other mechanisms in term of the accuracy. Through extensive experiments on synthetic data set and real world data set, we demonstrate that the performance of proposed mechanism is better.

The role of the Industrial Internet of Things (IIoT) in critical infrastructure sectors, such as power, chemistry, and manufacturing, will be increasingly important as we move toward Industry 5.0. For example, IIoT devices are deployed in factories to help the manufacturing companies (e.g., automotive) gain in-depth insight into the various states of production, and thus improving production efficiency and achieving cost reductions. However, malicious code may compromise IIoT devices if either the devices are exposed to outside or unexposed inner devices are updated unauthentically. Due to their limited resources and features, it is challenging to implement strong security solutions for such embedded devices. In this article, we propose a blockchain-based software status monitoring system, called BoSMoS. The system is designed to monitor the software status of IIoT devices to detect and respond to identified malicious behaviors (e.g., intrusions). BoSMoS takes a snapshot of the statue of monitored software and monitors its file system calls. In order to ensure the software integrity information, we use blockchain as the distributed ledger to store a snapshot of software status. The blockchain network of BoSMoS can employ different consensus algorithms. We also evaluate the performance of BoSMoS, in terms of exception response delay, resistance performance to various intrusions, and scalability. The experimental results justify that BoSMoS is practical and sound. In addition, the evaluation of scalability and security demonstrates that the system can carry deployment of large-scale IIoT devices and can guarantee authenticated software updating, as well as detect unauthorized software status.

Terminals with diverse technological specifications, heterogeneous network environment, and personalized user requirements raise new challenges to streaming media services. Solutions such as the newly standardized H.264/SVC (scalable video coding; designed to compress original video bitstream into a multilayer video stream according to requirements) have been proposed. With the pervasive application of SVC in applications, such as video on demand, video conferencing, and video surveillance in the Internet of Things (IoT), there has been increased scrutiny on security of H.264/SVC. In this article, we propose a bitstream-oriented layered encryption scheme for SVC bitstream. According to the multilayer bit code structure of SVC, the bitstream is separated and encrypted, respectively, by rearranging the network abstraction layer (NAL) unit of SVC bitstream. This provides hierarchical protection for the multilayer characteristic of SVC. In order to provide sufficient security, as well as achieving improved computational efficiency, we use different cryptographic algorithms for the base layer and enhancement layers according to its requirements. The base layer adopts off-the-shelf high-security encryption algorithms, such as block cipher, to ensure security. Each enhancement layer is encrypted with a different key through the stream cipher with low computational complexity, providing layered control of the video. Furthermore, we propose a hierarchical key management scheme to implement layered access control according to the principle of hierarchical deterministic wallet (H-D wallet). Our scheme can be applied to the user-level distinction in video on demand and video surveillance systems in IoT. The analysis and experiments indicate that the proposed scheme achieves a high-security level, yet incurs reasonably low compression cost and computational complexity.

Agent advising is one of the key approaches to improve agent learning performance by enabling agents to ask for advice between each other. Existing agent advising approaches have two limitations. The first limitation is that all the agents in a system are assumed to be friendly and cooperative. However, in the real world, malicious agents may exist and provide false advice to hinder the learning performance of other agents. The second limitation is that the analysis of communication overhead in these approaches is either overlooked or simplified. However, in communication-constrained environments, communication overhead has to be carefully considered. To overcome the two limitations, this paper proposes a novel differentially private agent advising approach. Our approach employs the Laplace mechanism to add noise on the rewards used by student agents to select teacher agents. By using the differential privacy technique, the proposed approach can reduce the impact of malicious agents without identifying them. Also, by adopting the privacy budget concept, the proposed approach can naturally control communication overhead. The experimental results demonstrate the effectiveness of the proposed approach.

Privacy preserving in machine learning is a crucial issue in industry informatics since data used for training in industries usually contain sensitive information. Existing differentially private machine learning algorithms have not considered the impact of data correlation, which may lead to more privacy leakage than expected in industrial applications. For example, data collected for traffic monitoring may contain some correlated records due to temporal correlation or user correlation. To fill this gap, in this article, we propose a correlation reduction scheme with differentially private feature selection considering the issue of privacy loss when data have correlation in machine learning tasks. The proposed scheme involves five steps with the goal of managing the extent of data correlation, preserving the privacy, and supporting accuracy in the prediction results. In this way, the impact of data correlation is relieved with the proposed feature selection scheme, and moreover the privacy issue of data correlation in learning is guaranteed. The proposed method can be widely used in machine learning algorithms, which provide services in industrial areas. Experiments show that the proposed scheme can produce better prediction results with machine learning tasks and fewer mean square errors for data queries compared to existing schemes.

The location privacy issue has become a critical research topic recently. The existing solutions do not solve one typical problem in practice: people may only want to protect certain privacy-sensitive locations among a group of temporal and spatial correlated points in a trajectory. As an effort towards this issue, we analyze the impact of space-time relationship on location privacy preservation. In addition, we propose new privacy definitions to better evaluate the privacy level and prove that the target location's privacy can be enhanced by randomizing its time and space related points. Moreover, under the constraint of the total noise power, the problem of obfuscating a location in a temporal and spatial correlated trajectory is formulated as finding the best noise allocation vector which can achieve the highest privacy level. This problem is solved by our proposed location privacy preserving method which applies differential privacy scheme on a series of points with noise budget allocation. Lastly, the performance of the proposed scheme is evaluated by simulations.

Artificial Intelligence (AI) has attracted a large amount of attention in recent years. However, several new problems, such as privacy violations, security issues, or effectiveness, have been emerging. Differential privacy has several attractive properties that make it quite valuable for AI, such as privacy preservation, security, randomization, composition, and stability. Therefore, this paper presents differential privacy mechanisms for multi-agent systems, reinforcement learning, and knowledge transfer based on those properties, which proves that current AI can benefit from differential privacy mechanisms. In addition, the previous usage of differential privacy mechanisms in private machine learning, distributed machine learning, and fairness in models is discussed, bringing several possible avenues to use differential privacy mechanisms in AI. The purpose of this paper is to deliver the initial idea of how to integrate AI with differential privacy mechanisms and to explore more possibilities to improve AIs performance.

Summary Deep learning methods have become the basis of new AI‐based services on the Internet in big data era because of their unprecedented accuracy. Meanwhile, it raises obvious privacy issues. The deep learning–assisted privacy attack can extract sensitive personal information not only from the text but also from unstructured data such as images and videos. In this paper, we proposed a framework to protect image privacy against deep learning tools, along with two new metrics that measure image privacy. Moreover, we propose two different image privacy protection schemes based on the two metrics, utilizing the adversarial example idea. The performance of our solution is validated by simulations on two different datasets. Our research shows that we can protect the image privacy by adding a small amount of noise that has a humanly imperceptible impact on the image quality, especially for images of complex structures and textures.

Summary In recent years, the ubiquity of mobile devices has made spatial crowdsourcing a successful business platform for conducting spatiotemporal projects. In spatial crowdsourcing, workers contribute to a project by performing a task at a specific location. However, these platforms present serious threats to people's location privacy because sensitive information may be leaked from submitted spatiotemporal data. As a result, people may be hesitant to join spatial crowdsourcing projects, which hampers further applications of this business model. In this paper, we propose a private spatial crowdsourcing data submission algorithm, called PS‐Sub. This is a differentially private method that preserves people's location privacy and provides acceptable data utility. Rigorous privacy analyses theoretically demonstrate the privacy guarantees inherent in the proposed model. Experiments based on real‐world datasets were conducted using practical evaluation metrics. The results show that our method is able to achieve location privacy preservation efficiently, at an acceptable cost for spatial crowdsourcing applications.

Today search engines are tightly coupled with social networks, and present users with a double-edged sword: they are able to acquire information interesting to users but are also capable of spreading viruses introduced by hackers. It is challenging to characterize how a search engine spreads viruses, since the search engine serves as a virtual virus pool and creates propagation paths over the underlying network structure. In this paper, we quantitatively analyze virus propagation effects and the stability of the virus propagation process in the presence of a search engine. First, although social networks have a community structure that impedes virus propagation, we find that a search engine generates a propagation wormhole. Second, we propose an epidemic feedback model and quantitatively analyze propagation effects based on a model employing four metrics: infection density, the propagation wormhole effect, the epidemic threshold, and the basic reproduction number. Third, we verify our analyses on four real-world data sets and two simulated data sets. Moreover, we prove that the proposed model has the property of partial stability. Evaluation results show that, compared the cases without a search engine, virus propagation with the search engine has a higher infection density, shorter network diameter, greater propagation velocity, lower epidemic threshold, and larger basic reproduction number.

The unprecedented accuracy of deep learning methods has earned themselves as the foundation of new AI-based services on the Internet. At the same time, it presents obvious privacy issues. The deep learning aided privacy attack can dig out sensitive personal information not only from the text but also from unstructured data such as images and videos. In this paper, we proposed a framework to protect image privacy against the deep learning tools. We also propose two new metrics to measure the image privacy. Moreover, we propose two different image privacy protection schemes based on the two metrics, utilizing the adversarial example idea. The performance of our schemes is validated by simulation on a large-scale dataset. Our study shows that we can protect the image privacy by adding a small amount of noise, while the added noise has a humanly imperceptible impact on the image quality.

As one of the most influential privacy definitions, differential privacy provides a rigorous and provable privacy guarantee for data publishing. However, the curator has to release a large number of queries in a batch or a synthetic dataset in the Big Data era. Two challenges need to be tackled: one is how to decrease the correlation between large sets of queries, while the other is how to predict on fresh queries. This paper transfers the data publishing problem to a machine learning problem, in which queries are considered as training samples and a prediction model will be released rather than query results or synthetic datasets. When the model is published, it can be used to answer current submitted queries and predict results for fresh queries from the public. Compared with the traditional method, the proposed prediction model enhances the accuracy of query results for non-interactive publishing. We prove that learning model can successfully retain the utility of published queries while preserving privacy.

Purpose The purpose of this study is to provide a framework to detect and prevent virtual property theft in virtual world environments. The issue of virtual property theft is a serious problem which has ramifications in both the real and virtual world. Virtual world users invest a considerable amount of time, effort and often money to collect virtual property, only to have them stolen by thieves. Many virtual property thefts go undetected and often only discovered after the incident has occurred. Design/methodology/approach This paper presents the design of an autonomic detection framework to identify virtual property theft at two key stages: account intrusion and virtual property trades. Account intrusion is an unauthorized user attempting to gain access to an account and unauthorized virtual property trades are trading of items between two users which exhibit theft characteristics. Findings Initial tests of this framework on a synthetic data set show an 80 per cent detection rate. This framework allows virtual world developers to tailor and extend it to suit their specific requirements. It provides an effective way of detecting virtual property theft while being low maintenance, user friendly and cost effective. Originality/value To the author’s knowledge, there is no detection framework, system or tool that works on virtual property theft detection in virtual world environments without access to authentic virtual world data or attack data (because of privacy issues and unwillingness of virtual world environments companies to collaborate). The topic of virtual property theft, lack of existing labelled data sets, user anonymity, size of virtual world environments data sets and privacy issues with virtual world companies and a number of other critical factors distinguish this paper from previous studies.

Differential privacy is an essential and prevalent privacy model that has been widely explored in recent decades. This survey provides a comprehensive and structured overview of two research directions: differentially private data publishing and differentially private data analysis. We compare the diverse release mechanisms of differentially private data publishing given a variety of input data in terms of query type, the maximum number of queries, efficiency, and accuracy. We identify two basic frameworks for differentially private data analysis and list the typical algorithms used within each framework. The results are compared and discussed based on output accuracy and efficiency. Further, we propose several possible directions for future research and possible applications.

Summary With the increasing use of location‐based services, location privacy has recently started raising serious concerns. Location perturbation and obfuscation are most widely used for location privacy preserving. To protect a user from being identified, a cloaked spatial region that contains otherk− 1 nearest neighbors of the user is submitted to the location‐based service provider, instead of the accurate position. In this paper, we consider the location‐aware applications that services are different among regions. In such scenarios, the semantic distance between users should be considered besides the Euclidean distance for searching the neighbors of a user. We define a novel distance measurement that combines the semantic and the Euclidean distance to address the privacy‐preserving issue in the aforementioned applications. We also present an algorithmkNNH to implement our proposed method. Moreover, we conduct performance study experiments on the proposed algorithm. The experimental results further suggest that the proposed distance metric and the algorithm can successfully retain the utility of the location services while preserving users' privacy. Copyright © 2015 John Wiley & Sons, Ltd.

Abstract Google Android is popular for mobile devices in recent years. The openness and popularity of Android make it a primary target for malware. Even though Android's security mechanisms could defend most malware, its permission model is vulnerable to transitive permission attack, a type of privilege escalation attacks. Many approaches have been proposed to detect this attack by modifying the Android OS. However, the Android's fragmentation problem and requiring rooting Android device hinder those approaches large‐scale adoption. In this paper, we present an instrumentation framework, called SEAPP, for Android applications (or “apps”) to detect the transitive permission attack on unmodified Android. SEAPP automatically rewrites an app without requiring its source codes and produces a security‐harden app. At runtime, call‐chains are built among these apps and detection process is executed before a privileged API is invoked. Our experimental results show that SEAPP could work on a large number of benign apps from the official Android market and malicious apps, with a repackaged success rate of over 99.8%. We also show that our framework effectively tracks call‐chains among apps and detects known transitive permission attack with low overhead. Copyright © 2016 John Wiley & Sons, Ltd.

Privacy preserving on data mining and data release has attracted an increasing research interest over a number of decades. Differential privacy is one influential privacy notion that offers a rigorous and provable privacy guarantee for data mining and data release. Existing studies on differential privacy assume that in a data set, records are sampled independently. However, in real-world applications, records in a data set are rarely independent. The relationships among records are referred to as correlated information and the data set is defined as correlated data set. A differential privacy technique performed on a correlated data set will disclose more information than expected, and this is a serious privacy violation. Although recent research was concerned with this new privacy violation, it still calls for a solid solution for the correlated data set. Moreover, how to decrease the large amount of noise incurred via differential privacy in correlated data set is yet to be explored. To fill the gap, this paper proposes an effective correlated differential privacy solution by defining the correlated sensitivity and designing a correlated data releasing mechanism. With consideration of the correlated levels between records, the proposed correlated sensitivity can significantly decrease the noise compared with traditional global sensitivity. The correlated data releasing mechanism correlated iteration mechanism is designed based on an iterative method to answer a large number of queries. Compared with the traditional method, the proposed correlated differential privacy solution enhances the privacy guarantee for a correlated data set with less accuracy cost. Experimental results show that the proposed solution outperforms traditional differential privacy in terms of mean square error on large group of queries. This also suggests the correlated differential privacy can successfully retain the utility while preserving the privacy.

Tagging recommender systems allow Internet users to annotate resources with personalized tags. The connection among users, resources and these annotations, often called a folksonomy, permits users the freedom to explore tags, and to obtain recommendations. Releasing these tagging datasets accelerat es both commercial and research work on recommender systems. However, tagging recommender systems has been confronted with serious privacy concerns because adversaries may re-identify a user and her/his sensitive information from the tagging dataset using a little background information. Recently, several private techniques have been proposed to address the problem, but most of them lack a strict privacy notion, and can hardly resist the number of possible attacks. This paper proposes an private releasing algorithm to perturb users’ profile in a strict privacy notion, differential privacy, with the goal of preserving a user’s identity in a tagging dataset. The algorithm includes three privacy-preserving operations: Private Tag Clustering is used to shrink the randomized domain and Private Tag Selection is then applied to find the most suitable replacement tags for the original tags. To hide the numbers of tags, the third operation, Weight Perturbation, finally adds Laplace noise to the weight of tags. We present extensive experimental results on two real world datasets, De.licio.us and Bibsonomy. While the personalization algorithm is successful in both cases, our results further suggest the private releasing algorithm can successfully retain the utility of the datasets while preserving users’ identity.

As a popular technique in recommender systems, Collaborative Filtering (CF) has received extensive attention in recent years. However, its privacy-related issues, especially for neighborhood-based CF methods, can not be overlooked. The aim of this study is to address the privacy issues in the context of neighborhood-based CF methods by proposing a Private Neighbor Collaborative Filtering (PNCF) algorithm. The algorithm includes two privacy-preserving operations: Private Neighbor Selection and Recommendation-Aware Sensitivity. Private Neighbor Selection is constructed on the basis of the notion of differential privacy to privately choose neighbors. Recommendation-Aware Sensitivity is introduced to enhance the performance of recommendations. Theoretical and experimental analysis are provided to show the proposed algorithm can preserve differential privacy while retaining the accuracy of recommendations.

Tagging recommender systems allow Internet users to annotate resources with personalized tags. The connection among users, resources and these annotations, often called afolksonomy, permits users the freedom to explore tags, and to obtain recommendations. Releasing these tagging datasets accelerates both commercial and research work on recommender systems. However, adversaries may re-identify a user and her/his sensitivity information from the tagging dataset using a little background information. Recently, several private techniques have been proposed to address the problem, but most of them lack a strict privacy notion, and can hardly resist the number of possible attacks. This paper proposes an private releasing algorithm to perturb users' profile in a strict privacy notion, differential privacy, with the goal of preserving a user's identity in a tagging dataset. The algorithm includes three privacy preserving operations: Private Tag Clustering is used to shrink the randomized domain and Private Tag Selection is then applied to find the most suitable replacement tags for the original tags. To hide the numbers of tags, the third operation, Weight Perturbation, finally adds Lap lace noise to the weight of tags We present extensive experimental results on two real world datasets, Delicious and Bibsonomy. While the personalization algorithmis successful in both cases.

Human associated delay-tolerant networks (HDTNs) are new networks where mobile devices are associated with humans and demonstrate social-related communication characteristics. Most of recent works use real social trace file to analyse its social characteristics, however social-related data is sensitive and has concern of privacy issues. In this paper, we propose an anonymous method that anonymize the original data by coding to preserve individual's privacy. The Shannon entropy is applied to the anonymous data to keep rich useful social characteristics for network optimization, e.g. routing optimization. We use an existing MIT reality dataset and Infocom 06 dataset, which are human associated mobile network trace files, to simulate our method. The results of our simulations show that this method can make data anonymously while achieving network optimization.

Traditional financial surveillance system usually discriminates suspicious transaction by comparing every transaction against its corresponding account history. This process always results in high false positive rate because it is regardless of the existence of economic cycle and business fluctuation. We conceived a new analyzingprototype by comparing an account time series transaction data against its peer group. It has deeply considered the injuence of normal jluctuation widely existing in real life and could eflciently reduce the number of false positives with a better understanding of customer 5. behavior pattern. A new method Empirical Mode Decomposition (EMD) developed initially for natural and engineering sciences has now been applied to financial time series data. This method has shown its superiorities in analyzing nonlinear and nonstationary stochastic engineering time series over traditional Discrete Fourier Decomposrtlon (DFD) and wavelet decomposition methods. Firstly the complex financial time series is decomposed into some local detail parts and one global tendency part which represent different time scales like daily, monthly, seasonal or annuat. Then a linear segment approximation method based on hierarchical piecewise linear representation (LPR) is used to fulfill the quick matching of the major tendency parts between two time series. The results from experiments on real life bank data lfoveign exchange transaction data sets) exhibit that the EMD can become a vital technique for the analysis offinancial suspicious transaction detection.

Outlier detection is a key element for intelligent financial surveillance systems which intend to identify fraud and money laundering by discovering unusual customer behaviour pattern. The detection procedures generally fall into two categories: comparing every transaction against its account history and further more, comparing against a peer group to determine if the behavior is unusual. The later approach shows particular merits in efficiently extracting suspicious transaction and reducing false positive rate. Peer group analysis concept is largely dependent on a cross-datasets outlier detection model. In this paper, we propose a new cross outlier detection model based on distance definition incorporated with the financial transaction data features. An approximation algorithm accompanied with the model is provided to optimize the computation of the deviation from tested data point to the reference dataset. An experiment based on real bank data blended with synthetic outlier cases shows promising results of our model in reducing false positive rate while enhancing the discriminative rate remarkably