Augmented Reality (AR) and Multimodal Large Language Models (LLMs) are rapidly evolving, providing unprecedented capabilities for human-computer interaction. However, their integration introduces a new attack surface for Social Engineering (SE). In this paper, we systematically investigate the feasibility of orchestrating AR-driven Social Engineering attacks using Multimodal LLM for the first time, via our proposed SEAR framework, which operates through three key phases: (1) AR-based social context synthesis, which fuses Multimodal inputs (visual, auditory and environmental cues); (2) role-based Multimodal RAG (Retrieval-Augmented Generation), which dynamically retrieves and integrates social context; and (3) ReInteract social engineering agents, which execute adaptive multiphase attack strategies through inference interaction loops. To verify SEAR, we conducted an IRB-approved study with 60 participants and build a novel dataset of 180 annotated conversations in different social scenarios (e.g., coffee shops, networking events). Our results show that SEAR is highly effective at eliciting high-risk behaviors (e.g., 93.3% of participants susceptible to email phishing). The framework was particularly effective in building trust, with 85% of targets willing to accept an attacker's call after an interaction. Also, we identified notable limitations such as authenticity gaps. This work provides proof-of-concept for AR-LLM driven social engineering attacks and insights for developing defenses against next-generation AR/LLM-based SE threats.

Retrieval-Augmented Generation (RAG) is a state-of-the-art technique that mitigates issues such as hallucinations and knowledge staleness in Large Language Models (LLMs) by retrieving relevant knowledge from an external database to assist in content generation. Existing research has demonstrated potential privacy risks associated with the LLMs of RAG. However, the privacy risks posed by the integration of an external database, which often contains sensitive data such as medical records or personal identities, have remained largely unexplored. In this paper, we aim to bridge this gap by focusing on membership privacy of RAG’s external database, with the aim of determining whether a given sample is part of the RAG’s database. Our basic idea is that if a sample is in the external database, it will exhibit a high degree of semantic similarity to the text generated by the RAG system. We present S2MIA, a Membership Inference Attack that utilizes the Semantic Similarity between a given sample and the content generated by the RAG system. With our proposed S2MIA, we demonstrate the potential to breach the membership privacy of the RAG database. Extensive experimental results demonstrate that S2MIA outperforms five existing MIAs, even when the system is protected by three representative defenses.

The improvement of transferability of adversarial examples is the key property in practical black-box scenarios. Recent research has identified that transferable adversarial examples for video models can be effectively crafted with image models. However, existing studies primarily target single-layer features, overlooking the influence of diverse feature layers. Moreover, they neglect transitions between video frames and fail to fully capture temporal context. In this paper, we introduce an efficient and stable cross-modal attack method termed Feature and Temporal Disruption Attack (FTDA). Our approach caters to both feature space diversity and temporal cues by introducing two innovative modules, i.e., Depth-Aware Feature Fusion Attack (DF2A) and Clip-Based Temporal Fusion Attack (CTFA). Extensive experiments demonstrate that our approach achieves SOTA. Our code is available at https://github.com/xiaopengge2000/FTDA.

Clustering Federated Learning (CFL) promotes knowledge sharing by adaptively grouping similar clients while solving the Non-IID problem to provide personalized models with high generalization. However, there are several problems needed to be considered before the practical deployment of CFL: 1) Practical clients often have limited communication capabilities. 2) Dynamically evolving data distributions of clients may leads to unreasonable clustering. 3) Asynchronization will cause clients belonging to the same cluster to fall apart. The above three problems will reduce the convergence speed and damage the accuracy of the model. In this work, we propose a Dynamic Representation-driven Clustering Federated Learning framework (DReCFL) to solve the above three problems. Specifically, DReCFL replaces complex model parameters with data representation for communicating to reduce the communication pressure. In order to gain a reasonable clustering, DReCFL utilizes a Dynamic Client Fuzzy Clustering (DCFC) algorithm, we proposed, to adapt to evolving data distributions. Finally, DReCFL leverages an Adaptive Clustering Threshold (ACT) mechanism, we designed, to craft clustering thresholds based on training progress, ensuring the reliability of client clustering results under asynchrony. Extensive experimental results demonstrate that DReCFL efficiently adapts to the addition of new clients and changes in data distribution, while reducing communication overhead by 22.70%-83.33% compared to state-of-the-art (SOTA) personalized federated learning methods.

Federated learning (FL) is a distributed learning paradigm that enables multiple clients to train a unified model without sharing their private data. However, recent works demonstrate that FL models are vulnerable to membership inference attacks (MIAs), which can infer whether a data sample was used to train a given FL model. Existing countermeasures either require far-reaching modifications of FL training process or enforce extra processing in prediction phase, yielding them unlikely to be applied well in practice. In this paper, we design a post-protection mechanism, dubbedP$^{2}$-Protection, which degrades the inference performance of MIAs by simultaneously poisoning the prediction and gradient of the target FL model to reduce the privacy leakage of training data while keeping the model prediction accuracy.P$^{2}$-Protectiononly involves one additional training round to embed the poisoned prediction and gradient into the target FL model, without requiring model retraining or training process modification. We evaluateP$^{2}$-Protectionand compare it with two state-of-the-art defenses against three MIAs on five realistic datasets. Experimental results show thatP$^{2}$-Protectionoutperforms the existing defenses by offering limited implement overhead and improved utility-privacy trade-off.

In the growing trend of remote working, millions of edge networks (e.g., homes or branch offices) are increasingly threatened by Advanced Persistent Threats (APTs), because of the weakened segmentation between business and non-business devices in remote working environment. Despite the fact that numerous APT detection mechanisms have been proposed, all of them are struggling to handle thecomplex structure, themassive scaleand thediverse topologyof edge networks.Can recent machine-learning advances tackle these APT detection pain points in edge networks?The GNNs (Graph Neural Networks) seems to be suited to capture thecomplex structure, but its adjacency matrix fails to capture key network flow context. Additionally, GNNs require extensive manual labeling, which is not scalable. LLMs (Large Language Models) have the potential to provide automatic labeling for the GNNs, but they lack the supplementary security context needed for effective labeling. To address these gaps, we presentLLMGraph, which incorporates extended GCNs (Graph Convolutional Networks) and domain-specific RAG (Retrieval-Augmented Generation) pipeline to achieve label-free detection against APTs in edge networks.LLMGraph's extended GCNs model can capture network flow context and direction.LLMGraph's domain-specific RAG pipeline can supplement key security contexts, including device vulnerability and network flow, for effective labeling. Additionally,LLMGraphprovides an LLM aggregator to augment and merge thediverse topologyof the edge networks. Compared to the state-of-the-art mechanisms,LLMGraphproveseffectiveandscalable, improving the F1-score by at least 46.9%, and the training time for 1 million edge networks is within 1000 s.

Clustered Federated Learning (CFL) effectively mitigates the negative impact of data heterogeneity among clients on model accuracy by grouping clients with similar data distributions for training. However, existing methods require communication of model parameters between clients and the server, leading to high communication costs. Additionally, current clustering strategies restrict the flexibility of cluster partitioning. To address these issues, we propose a Dynamic Representation Clustering for Federated Learning (DRCFL) algorithm. It decouples the model into a Representation Generation Module (RGM) and a Shared header (S-header). The former calculates client data representations, while the server clusters these representations and trains a shared header for each cluster. Clients only need to transmit data representations to the server, remarkably reducing communication costs. Additionally, the server re-clusters based on the most recent data representations in each training round, enhancing the flexibility of cluster partitions. In terms of communication cost, Extensive experimental results indicate that DRCFL achieves a reduction of 22.70%−83.33% compared to state-of-the-art (SOTA) personalized FL methods.

In the domain of computer vision, Camouflaged Object Segmentation (COS) is a crucial task aimed at identifying objects that blend into their surroundings, with applications spanning diverse sectors such as military, medical, and beyond. Traditional COS techniques, which primarily depend on supervised learning, necessitate large-scale labeled datasets. However, the acquisition of sufficient camouflage images for such purposes is often constrained due to their scarcity and the high cost of manual annotation. Additionally, these conventional methods frequently struggle to generalize to novel, unseen classes. In response to these challenges, this paper proposes the Camouflage Few-Shot (CAMFS) framework, an innovative approach integrating few-shot learning into COS. The CAMFS framework comprises two main components: the Camouflaged-Meta module, which converts the semantic information of camouflaged objects into compact feature vectors to facilitate knowledge transfer from support to query images; and the Camouflaged-Base module, focused on refining edge detection and enhancing the contrast between foreground and background elements. To overcome the limitations of existing COS datasets, which are primarily designed for supervised learning, we have developed the COS-FSS dataset, the first public few-shot COS dataset. It is based on the COD10K dataset and supplemented with approximately 3200 additional camouflage images. We conducted extensive evaluations of our CAMFS framework on the COS-FSS dataset. Compared to existing COS models, CAMFS demonstrates an average improvement of 5.7% in Sαand 14.02% in $F_\beta ^w$, while against few-shot segmentation models, it achieves a 5.97% increase in m-IoU. The dataset and additional resources are available at https://github.com/CAM-FSS/FSS-COD.

In Federated Learning (FL), the regulatory need for the "right to be forgotten" requires efficient Federated Unlearning (FU) methods, which enable FL models to unlearn appointed training data. Associating with the emergence FU, verifying the performance of FU plays a critical role in evaluating the consistency FU methods, in case of the unexpected degradation of the FL model. Though well developed, none of the existing verification methods in FU stands for the clients who opt out of the FL process, which is a universal demand in FL. More specifically, after the clients quit the FL cooperation, they can no longer verify whether the FL model unlearns their data after the FL keeps training for several rounds. To this end, we introduce a continuous verification mechanism for FL clients, called Backdoor Attack-based Forgetting Verification (BAFV). Inspired by backdoor attack, BAFV embedded a persistent mark for the client that proposes to leave, with the intention that the client still has the right to verify of FU after leaving the FL cooperation for a relatively long period. Extensive experiments across diverse FU environments and datasets demonstrated that our method maintains the accuracy of model and provides clients with a continuous verification mechanism to defend their rights. Our code of BAFV is publicly available at: https://github.com/paper-liu/BAFV-master.git.

Federated Learning is a distributed model training system that uses the computing resources and private data of participants to collaboratively train Machine Learning models. In order to incentivize data holders to actively participate in Federated Learning, a crucial issue is how to fairly assess each participant’s contribution. One class of the mainstream methods estimates client contributions by calculating the correlation between local and global model parameters, substantially reducing the reliance on large computing resources and test datasets. However, these methods overlook an implicit key factor, leading to inaccuracy in assessing the contribution of the participants. We propose FedCA, which provides a fairness assessment of the client’s contribution based on the performance gain of the global model in each communication round. The experimental results demonstrate that FedCA accurately identifies the contributions of clients with varying data qualities and effectively approximates the optimal fairness method of Shapley values. The source code for this paper is available at https://github.com/paper-liu/FedCA-master.git.

In recent years, deep neural networks (DNNs) have witnessed extensive applications, and protecting their intellectual property (IP) is thus crucial. As a non-invasive way for model IP protection, model fingerprinting has become popular. However, existing single-point based fingerprinting methods are highly sensitive to the changes in the decision boundary, and may suffer from the misjudgment of the resemblance of sparse fingerprinting, yielding high false positives of innocent models. In this paper, we propose ADV-TRA, a more robust fingerprinting scheme that utilizes adversarial trajectories to verify the ownership of DNN models. Benefited from the intrinsic progressively adversarial level, the trajectory is capable of tolerating greater degree of alteration in decision boundaries. We further design novel schemes to generate a surface trajectory that involves a series of fixed-length trajectories with dynamically adjusted step sizes. Such a design enables a more unique and reliable fingerprinting with relatively low querying costs. Experiments on three datasets against four types of removal attacks show that ADV-TRA exhibits superior performance in distinguishing between infringing and innocent models, outperforming the state-of-the-art comparisons.

The location context can benefit a broad range of context-aware applications, where recognizing and locating contextual objects, such as hair dryers, coffee machines, or water faucets, which are not equipped with any smart modules and thus unable to emit modulated signals, provide fine-grained contextual information. While there have been extensive researches on localizing smart mobile devices, little has been done for locatingcontextual objects, let alone for recognizing and locating them together. In this article, we aim to study the problem of simultaneously recognizing and locating such contextual objects and present CORAL, a contextual object recognition and locating scheme by the usage of unmodulated acoustic signals from the working contextual objects recorded by the commercial off-the-shelf smartphones of users. Specifically, CORAL exploits the frequency and power features of these signals to build a mel-frequency cepstral coefficients (MFCCs) data set for contextual objects, and constructs a classifier for contextual object recognition by using bidirectional LSTM (BiLSTM) and a regression model for object-to-device distance computation by using LightGBM, which is then used for object locating with the help of the user’s trace. We implement a prototype of CORAL and extensive experiments show that the CORAL achieves high recognition accuracy and locating accuracy, even when there are concurrent working contextual objects or ambient noises.

Point-of-interest (POI) recommendation has drawn much attention with the widespread popularity of location-based social networks (LBSNs). Previous works define long- and short-term trajectories via long short-term memory (LSTM) to capture user's stable and current preference, and incorporate context factors to improve recommendation effectiveness. However, these factors have different impacts on POI recommendation, and meanwhile, they are mutually influenced. Existing studies either model all the factors separately, or feed them into the same LSTM model, which are less meticulous for modeling the LBSNs trajectories. To address such issues, we revisit the long- and short-term preference learning for next POI recommendation by presenting a novel framework that can model both POI level and semantic level check-in trajectories. We develop a hierarchical LSTM to learn the two-level representations and consider the interplay of the two-level features by adding factors to the gates of LSTMs for each trajectory. We further construct a semantic filter to improve the recommendation efficacy. Experimental results using two real-world check-in datasets indicate that the proposed framework outperforms four state-of-the-art baselines regarding two commonly used metrics.

Driven by the large amount of spatio-temporal data obtained from location-based social networks, the implementation of cross-domain user linkage, also known as the User Identity Linkage (UIL), has attracted increasing research attentions. While most of the existing UIL works discretize the spatio-temporal sparse data when identifying encountering or co-located events for UIL, user’s distinctive behavior patterns implicit in the “check-in” spatio-temporal data with continuous nature pave the way for enhancing UIL performance. In this paper, we propose an approach dubbedCP-Linkthat exploits user behavior patterns in a continuous way. In CP-Link, the continuous space is divided into irregularly shaped stay regions, and a continuous time-based improved dynamic time warping (IDTW) method is proposed to calculate the similarity. To bridge the gap between the ideal scenario with ample records and the reality with sparse data, we adopt the user-associated location frequent pattern (LFP) model to compensate for the sparse deficiency. Extensive experiments conducted on real-world datasets demonstrate the effectiveness and superiority of CP-Link, which outperforms the state of the arts by more than 20% in terms of the AUC.

Federated learning (FL) has recently emerged as a promising distributed machine learning (ML) paradigm. Practical needs of the "right to be forgotten" and countering data poisoning attacks call for efficient techniques that can remove, or unlearn, specific training data from the trained FL model. Existing unlearning techniques in the context of ML, however, are no longer in effect for FL, mainly due to the inherent distinction in the way how FL and ML learn from data. Therefore, how to enable efficient data removal from FL models remains largely under-explored. In this paper, we take the first step to fill this gap by presenting FedEraser, the first federated unlearning method-ology that can eliminate the influence of a federated client’s data on the global FL model while significantly reducing the time used for constructing the unlearned FL model. The basic idea of FedEraser is to trade the central server’s storage for unlearned model’s construction time, where FedEraser reconstructs the unlearned model by leveraging the historical parameter updates of federated clients that have been retained at the central server during the training process of FL. A novel calibration method is further developed to calibrate the retained updates, which are further used to promptly construct the unlearned model, yielding a significant speed-up to the reconstruction of the unlearned model while maintaining the model efficacy. Experiments on four realistic datasets demonstrate the effectiveness of FedEraser, with an expected speed-up of 4× compared with retraining from the scratch. We envision our work as an early step in FL towards compliance with legal and ethical criteria in a fair and transparent manner.

Internet of Things (IoT) applications bring in a great convenience for human’s life, but users’ data privacy concern is the major barrier toward the development of IoT.${k}$-anonymity is a method to protect users’ data privacy, but it is presently known to suffer from inference attacks. Thus far, existing work only relies on a number of experimental examples to validate${k}$-anonymity’s performance against inference attacks, and thereby lacks of a theoretical guarantee. To tackle this issue, in this paper we propose the first theoretical foundation that gives a nonasymptotic bound on the performance of${k}$-anonymity against inference attacks, taking into consideration of adversaries’ background information. The main idea is to first quantify adversaries’ background information, and from the point of the view of adversaries, classify users’ data into four kinds: 1) independent with unknown data values; 2) local dependent with unknown data values; 3) independent with certain known data values; and 4) local dependent with certain known data values. We then move one step further, theoretically proving the bound on the performance of${k}$-anonymity corresponding to each of the four kinds of users’ data through cooperating with the noiseless privacy. We argue that such a theoretical foundation links${k}$-anonymity with noiseless privacy, theoretically proving${k}$-anonymity provides noiseless privacy. Additionally, this paper theoretically explains why${k}$-anonymity is vulnerable to inference attacks using the modified Stein method. Simulations on real check-in dataset from the location-based social network have validated our results. We believe that this paper can bridge the gap between design and evaluation, enabling a designer to construct a more practical${k}$-anonymity technique in real-life scenarios to resist inference attacks.

We consider the navigation application of 3D sensor networks that can proactively guide the movement of internal users from potential dangers to a safe exit, where a 3D sensor network serves as a reactive system, instead of a monitoring tool or a medium of data acquisition. Most if not all existing efforts in this line concentrate on 2D cases only, and none of them can be readily applied to 3D sensor networks, posing it a non-trivial challenge to design an effective and light-weight navigation protocol in 3D sensor networks. In this paper, we propose the first location-free, distributed, and scalable navigation protocol that can provide a navigation route for users inside the 3D sensor network with guaranteed safety. More specifically, we formulate the navigation problem as the minimum cumulative exposure problem, and design SNP, a navigation protocol based on the so-called 1-manifold skeleton, which offers a safe path with a near-optimal cumulative exposure to dangers. Extensive simulations validate the effectiveness and efficiency of the proposed algorithm.

Device-to-device (D2D) communication has been recently proposed to mitigate the burden of base stations by leveraging the underutilized cellular spectrum resources, where high overall network throughput and D2D access rate are critical for its service performance and availability. In this paper, we study the resource allocation problem to push the limit of D2D communication underlaying cellular networks by allowing multiple D2D links to share resource with multiple cellular links. We propose FRESH, afullresourcesharing scheme where each subchannel can be shared by a cellular link and an arbitrary number of D2D links. In particular, FRESH first divides the communication links into so-called full resource sharing sets such that, within each set, all D2D link members are able to reuse the whole allocated resources. Thereafter, it allocates a sum of spectrum resources to each obtained full resource sharing set. As compared with state-of-the-art schemes, FRESH provides fine-grained resource allocation, resulting in throughput improvements of up to one order of magnitude, and D2D access rate improvements of up to 5 times with a moderate node density (e.g., on the order of 1 user per 400 square meters).

The in-network data storage and retrieval are fundamental functions of sensor networks. Among many proposals, geographical hash table GHT is perhaps most appealing as it is very simple yet powerful with low communication cost, where the key is to correctly define the bounding box. It is envisioned that the skeleton has the power to facilitate computing a precise bounding box. In existing works, the focus has been on skeleton extraction algorithms targeting for 2D sensor networks, which usually deliver a 1-manifold skeleton consisting of 1D curves. It faces a set of non-trivial challenges when 3D sensor networks are considered, in order to properly extract the surface skeleton composed of a set of 2-manifolds and possibly 1D curves. In this paper, we study the problem of surface skeleton extraction in 3D sensor networks. We propose a scalable and distributed connectivity-based algorithm to extract the surface skeleton of 3D sensor networks. First, we propose a novel approach to identifying surface skeleton nodes by computing the extended feature nodes such that it is robust against boundary noise, etc. We then find the maximal independent set of the identified skeleton nodes and triangulate them to form a coarse-grained surface skeleton, followed by a refining process to generate the fine-grained surface skeleton. Furthermore, we design an efficient updating scheme to react to the network dynamics caused by node failure, insertion, etc. We also investigate the impact of boundary incompleteness and present a scheme to extract the surface skeleton under incomplete boundary. Finally, we apply the extracted surface skeleton to facilitate the design of data storage protocol and curve skeleton extraction algorithm. Extensive simulations show the robustness of the proposed algorithm to shape variation, node density, node distribution, communication radio model and boundary incompleteness, and its effectiveness for data storage and retrieval application with respect to load balancing.

In sensor networks, skeleton extraction has emerged as an appealing approach to support many applications such as load-balanced routing and location-free segmentation. While significant advances have been made for 2D cases, so far skeleton extraction for 3D sensor networks has not been thoroughly studied. In this paper, we conduct the first work of a unified framework providing a connectivity-based and distributed solution forline-likeskeleton extraction in both 2D and 3D sensor networks. We highlight its practice as: 1) it has linear time/message complexity; 2) it provides reasonable skeleton results when the network has low node density; 3) the obtained skeletons are robust to shape variations, node densities, boundary noise and communication radio model. In addition, to confirm the effectiveness of the line-like skeleton, a 3D routing scheme is derived based on the extracted skeleton, which achieves balanced traffic load, guaranteed delivery, as well as low stretch factor.

In-network data storage and retrieval are fundamental functions of sensor networks. Among many proposals, geographical hash table (GHT) is perhaps most appealing as it is very simple yet powerful with low communication cost, where the key is to correctly define the bounding box. It is envisioned that the skeleton has the power to facilitate computing a precise bounding box. In existing works, the focus has been on skeleton extraction algorithms targeting for 2D sensor networks, which usually delivers a 1-manifold skeleton consisting of 1D curves. It faces a set of non-trivial challenges when 3D sensor networks are considered, in order to properly extract the surface skeleton composed of a set of 2-manifolds and possibly 1D curves.

In sensor networks, skeleton extraction has emerged as an appealing approach to support many applications such as load-balanced routing and location-free segmentation. While significant advances have been made for 2D cases, so far skeleton extraction for 3D sensor networks has not been thoroughly studied. In this paper, we conduct the first work on the skeleton extraction in 3D sensor networks, and propose a unified framework for line-like skeleton extraction in both 2D and 3D sensor networks. Our algorithm has the following three steps: first, each node identifies itself as a skeleton node if the geodesic shortest paths between its nearest boundary nodes (referred to as feature nodes) decompose the boundary of the network into more than one connected component; second, each skeleton node is assigned a monotonically increasing importance measure according to the maximum Lebesgue measure of the connected components of the boundary such that the identified skeleton nodes are self-connected; and finally, the skeleton is pruned based on the proposed metric branch similarity. The proposed algorithm is connectivity-based, distributed and of low complexity. Extensive simulations show that it is robust to shape variations and boundary noise.

In sensor networks, skeleton (also known as medial axis) extraction is recognized as an appealing approach to support many applications such as load-balanced routing and location free segmentation. Existing solutions in the literature rely heavily on the identified boundaries, which puts limitations on the applicability of the skeleton extraction algorithm. In this paper, we conduct the first work of a connectivity-based and boundary free skeleton extraction scheme, in sensor networks. In detail, we propose a simple, distributed and scalable algorithm that correctly identifies a few skeleton nodes and connects them into a meaningful representation of the network, without reliance on any constraint on communication radio model or boundary information. The key idea of our algorithm is to exploit the necessary (but not sufficient) condition of skeleton points: the intersection area of the disk centered at a skeleton point x should be the largest one as compared to other points on the chord generated by x, where the chord is referred to as the line segment connecting x and the tangent point in the boundary. To that end, we present the concept of \epsilon-centrality of a point, quantitatively measuring how "central" a point is. Accordingly, a skeleton point should have the largest value of \epsilon-centrality as compared to other points on the chord generated by this point. Our simulation results show that the proposed algorithm works well even for networks with low node density or skewed nodal distribution, etc. In addition, we obtain two by-products, the boundaries and the segmentation result of the network.