With the advancement of artificial intelligence, vision-language models (VLMs) that integrate text and image modalities have become central to multimodal learning and are increasingly deployed in Internet of Things (IoT) environments such as smart surveillance, autonomous driving and industrial monitoring. However, VLMs are also highly susceptible to adversarial attacks. Most previous black-box attack methods towards the VLMs rely on either the soft label or substitute models. However, most of them need detailed information on the target model, which is often unavailable for real-world threat models due to limitations on additional access and resource restrictions. In this study, we propose a surrogate-free hard-label attack towards the VLMs, which does not require a substitute model, called Hard-Label Fusion Attack towards Vision-Language models. HFAVL directly leverages the model’s final decision to jointly generate the perturbation for both text and image modalities instead of the confidence score necessary for the soft label attack. By transforming the text into a continuous embedding space that enables them to be aligned with the image in a unified manner, HFAVL applies cooperative Monte Carlo-based estimation on the multimodal gradient. Aiming to improve the efficiency of attacks, we introduce a Mutual Iterative Refinement mechanism (MIR), which gradually searches for a better adversarial example through small iterative steps that blend perturbations across both modalities. Additionally, to alleviate the imbalance between textual and visual noise during the attack, we propose a new multimodal edit distance evaluation metric to measure the quality of multimodal adversarial examples. Extensive experimental results demonstrate that HFAVL consistently outperforms existing multimodal adversarial attack methods across various vision-language pretraining (VLP) models. Notably, under the setting with recall@10, HFAVL achieves over 80% attack success rate on most benchmarks, demonstrating its efficiency and high precision in generating effective adversarial examples.

Backdoor attacks pose severe security challenges to federated learning systems due to their stealthy nature. Existing detection methods primarily focus on identifying anomalies by analyzing discrepancies in client model updates. However, in federated learning, the non-independent and identically distributed (non-IID) nature of client data leads to inconsistencies among local model updates, which can mask the distinguishing features of backdoor attacks and consequently degrade the performance of detection methods. Unlike benign models, which are trained solely for a single classification task, backdoored models are simultaneously optimized for both the classification (main) task and the backdoor task. Therefore, training the backdoored models can be regarded as a multi-task learning problem. Inspired by information bottleneck theory, we observe that backdoored models exhibit more stable feature representations than benign models when performing the main task. Based on this insight, we propose a novel stability metric that quantitatively captures the disparity in feature map stability between backdoored and benign models. Leveraging this metric, we develop a new backdoor detection framework for federated learning. Our method computes anomaly scores for each client and selectively aggregates models with benign characteristics, effectively defending against backdoor attacks. We validate our approach through extensive experiments on multiple benchmark datasets under non-IID settings. The results demonstrate that our method consistently achieves high detection performance across a range of backdoor scenarios and data heterogeneity levels.

Link prediction serves as a fundamental task in graph-based applications, where graph neural networks (GNNs) are extensively applied to estimate node connectivity likelihood. However, GNN-based methods for homogeneous graphs encounter the semantic mixing issue in heterogeneous graphs. Previous works leverage the disentangled-based model to separate the semantic information into different factors and conduct the message passing for link prediction. However, their models suffer from information loss and inadequate expression, which harms link prediction performance. To address these limitations, we propose a language-assisted semantic-aware disentangled method for link prediction on heterogeneous graphs. First, we employ a factor-wise attention mechanism to reduce the information loss caused by the disentangled model. Specifically, we design a factor selection strategy to select disentangled factors and combine them to utilize more semantic information. Second, a language-graph learning method is developed to enhance contextual expression by fusing the features of nodes and edge textual information. Extensive experiments show that the proposed method outperforms existing state-of-the-art baselines.

As a well-known graph embedding method, Graph Convolutional Networks (GCNs) have been widely applied to recommendation systems and social media analysis, in which privacy concerns regarding sensitive data have emerged in the public view due to the collection of personal preferences. Although the regularization methods are introduced to improve the network's security, the GCN tends to memorize individual user information in latent representations susceptible to the Membership Inference Attack (MIA). In addition, the previous works focus on improving the security while hurting the utility, or vice versa, which induces “negative transfer”. In this paper, we propose a novel disentangled-based framework to defend MIA and alleviate the issue of negative transfer in multi-task learning. First, we divide the sensitive and practical channels from the latent representations of graph nodes to minimize their linear dependency. Then, to effectively train our model, we employ the sub-computational graphs to generate local gradients for different tasks and allocate losses to them. Finally, we propose a novel mixed updating strategy to accumulate the updating information of sub-computational graphs. Extensive experiments show that the proposed method can mitigate the risk of membership inference while ensuring model accuracy.

The rapid growth of industrial Internet of Things (IoT) devices presents significant opportunities for federated learning to advance secure machine learning in industrial IoT systems. However, the performance of global federated learning models deteriorates significantly due to data heterogeneity inherent in industrial IoT systems. To address this issue, we propose a novel personalized federated learning framework via fine-grained one-shot clustering (FedFOC). Our method simultaneously considers the dataset structure and the relative positions of samples to extract compact features for clustering. Specifically, we combine clients’ locally calculated sample-level distances with MinHash signatures of their label sets to transform their data distributions into 1-D vectors to reduce communication costs. These vectors are then clustered to identify the fine-grained similarities between clients’ data distributions. To capture the complex relationships among local data distributions, we perform soft clustering in a one-shot manner and quantify client cluster association strengths. Subsequently, the clustering results are utilized to improve the local models’ personalized performance. Extensive experiments conducted on four widely used public datasets under various heterogeneous scenarios demonstrate that FedFOC outperforms state-of-the-art personalized federated learning methods in the accuracy of local models in almost all scenarios partitioned by Dirichlet. In addition, we also validate the effectiveness of our approach on the industrial dataset, indicating that our method has potential value in real-world applications.

Abstract Centralized machine learning methods require the aggregation of data collected from clients. Due to the awareness of data privacy, however, the aggregation of raw data collected by Internet of Things (IoT) devices is not feasible in many scenarios. Federated learning (FL), a kind of distributed learning framework, can be running on multiple IoT devices. It aims to resolve the issues of privacy leakage by training a model locally on the client-side, other than on the server-side that aggregates all the raw data. However, there are still threats of poisoning attacks in FL. Label flipping attacks, typical data poisoning attacks in FL, aim to poison the global model by sending model updates trained by the data with mismatched labels. The central parameter aggregation server is hard to detect the label flipping attacks due to its inaccessibility to the client in a typical FL system. In this work, we are motivated to prevent label flipping poisoning attacks by observing the changes in model parameters that were trained by different single labels. We propose a novel detection method called average weight of each class in its associated fully connected layer. In this method, we detect label flipping attacks by identifying the differences of classes in the data based on the weight assignments in a fully connected layer of the neural network model and use the statistical algorithm to recognize the malicious clients. We conduct extensive experiments on benchmark data like Fashion-MNIST and Intrusion Detection Evaluation Dataset (CIC-IDS2017). Comprehensive experimental results demonstrated that our method has the detection accuracy over 90% for the identification of the attackers flipping labels.

The operation frequency of memristor seriously affects its applications in high-frequency working conditions. In this article, a new method using a sine-based function instead of the linear term in memristor models to spread the operation frequency ranges of memristor emulators is proposed. Via using this method on a flux-controlled memristor with smooth continuous cubic function as an example, research shows that the operation frequency range of this improved flux-controlled memristor is spread greatly. In particular, the improved memristor can still display an excellent pinched hysteresis loop even when the operation frequency is up to a few gigahertz, which is higher than most of the previously reported memristor emulators. The typical dynamic characteristics and the operation frequency range of the improved memristor are studied theoretically and numerically. The circuit simulations and physical experiments agree well with the theoretical analyses, which together demonstrate the effectiveness of the proposed methodology. Then, the improved memristor is applied to realize a chaotic oscillator to show its practical applications. In addition, the new method has universality, and it can be used by many other existing memristor emulators for spreading their operation frequency ranges.