Device-free Wi-Fi localization is a promising technology to localize users who do not carry smart devices. The basic idea is to separate and analyze the signals reflected off human body from the multi-path signals. However, previous works could only localize moving users, because they can not distinguish the signals reflected off static users or objects like walls and furniture. This paper presents the Respiration Localization system,RespLoc, which for the first time enables device-free Wi-Fi localization system for static users. To recognize static users, the key insight is that people can breathe but objects cannot. However, it is non-trivial to extract user’s location from the respiration signal, because the respiration signal is significantly weaker than the regular motion signal. To this end, we propose the equivalent analysis method. Instead of using traditional signal separation, which suffers from severe noise due to residual signal components, we propose to construct an equivalent signal with the following properties: First, the equivalent signal follows the same variation law as the respiration signal; Second, the equivalent signal is not affected by irrelevant static signals. Based on this equivalent signal, we are able to resolve location features from the multi-path signals directly without separating them. We implementRespLocon commodity Wi-Fi devices, and extensive experimental results demonstrate thatRespLoccan localize static users with a median error of 0.89 meters.

Massive MIMO has emerged as a cornerstone technology for 5G-Advanced and future 6G networks, yet its practical deployment remains limited by hardware cost and power consumption. Switch-based architectures, which share a small number of RF chains among many antenna elements, provide a scalable alternative, but create a new bottleneck: only a subset of antennas is observable at any given moment, leaving the channel state of the remaining elements unknown. Lacking this information prevents the system from exploiting advanced physical-layer functions such as digital beamforming or multi-stream MIMO. In this paper, we present ARGUS, a generative channel reconstruction framework that infers the CSI of unobserved antennas from partial observations. The key idea is that all antenna responses are governed by the same underlying wireless propagation environment, enabling the task to be formulated as a generative inference problem. We employ a variational autoencoder to capture the latent spatial structure and reconstruct unobserved channels through sampling. Extensive experiments show that our reconstructed CSI incurs less than 2.5% achievable rate loss, and real-world measurements demonstrate a more than 90% antenna-selection match rate, confirming the practicality of the proposed approach.

The dynamic nature of tasks and environments in Internet of Things (IoT) systems require deep learning models to continuously retrain on evolving data to ensure their effectiveness. Existing continual learning (CL) methods aim to mitigate catastrophic forgetting, where the model loses knowledge of previous tasks when learning new ones. However, these methods often ignore the memory resource competition caused by the parallel execution of multiple applications, which limits the realworld IoT application of CL in resource-constrained edge devices. In this article, we propose EDCL, a novel approach that enhances the training efficiency and model accuracy of CL methods while ensuring the uninterrupted operation of high-priority inference programs. Specifically, we first implement a custom batch sampler that can dynamically load batches and measure the memory usage and training time recorded via offline profiling. In the online stage, by monitoring the resource consumption of high-priority programs, EDCL can dynamically select batch policies that meet resource constraints and facilitate efficient training. Additionally, we propose an adaptive hierarchical buffer swap method to enhance the model’s ability to retain previously learned knowledge and mitigate forgetting. Extensive experiments show that EDCL effectively balances training efficiency and model accuracy while preventing high-priority inference programs from failing due to memory contention, demonstrating promising performance compared to baselines.

Wi-Fi device-free tracking enables the inference of user behaviors without physical contact, which is crucial for intelligent indoor location-based services. Nevertheless, the practical implementation of current tracking systems is constrained by several critical limitations: 1) The low-quality sensing signals in complex scenarios lead to increased tracking errors; 2) Existing methods inadequately adjust to dynamic environments, necessitating additional data collection or retraining processes. To address these challenges, this paper introduces EOC-Tracking, a device-free Wi-Fi tracking system that dynamically incorporates environmental information. Our key innovation involves leveraging obstacles to correct illogical users' trajectories and facilitate adjustment to varying environments. This significantly improves the accuracy of the follow-up in complex and changing environments. The EOC-Tracking system is built upon three fundamental design principles: 1) A lightweight dual-branch neural network architecture that effectively fuses environmental data with Wi-Fi signal characteristics; 2) An autonomous map updating mechanism that facilitates real-time adaptation to environmental layout modifications without human intervention; 3) A sophisticated data-driven, phased training paradigm that optimizes the model's ability to learn and apply obstacle constraints. We implement EOC-Tracking using commercial Wi-Fi devices and deploy it on low-power embedded systems such as the MCU. Experimental results demonstrate that EOC-Tracking can reduce tracking errors by at most 49.48% compared to datadriven methods and 62.21% compared to model-based methods in various complex scenarios.

Millimeter-wave (mmWave) 5G-V2X communications face significant challenges in real-time beam alignment within high-mobility vehicular networks. While environmentaware beam prediction methods mitigate channel estimation overhead, their efficacy is severely compromised by modality data loss stemming from lighting variations, adverse weather, or sensor failures. To address this issue, we propose a Cross-modal Losstolerant Beam Prediction model (CLBP). CLBP robustly fuses RGB camera and LiDAR data, employing a novel cross-modal attention mechanism to achieve resilient feature alignment across these heterogeneous modalities. Furthermore, a Branch Features Dynamic Fusion (BFDF) module adaptively reweights modality features, suppressing noise from degraded inputs and promoting effective information propagation to enhance resilience. To facilitate realistic evaluation, we introduce a Data-Conditioned Missingness Mechanism (DCMM), which augments the DeepSense 6G V2V dataset with meticulously simulated sensor failure scenarios. Experimental results demonstrate CLBP's superior performance, achieving 94.48% Top-5 beam prediction accuracy even under 10% modality loss, and a 29% reduction in average power loss compared to baseline methods. These findings demonstrate CLBP's significant robustness in dynamic vehicular environments and its capacity to maintain consistent, high-performance beam prediction despite challenging data imperfections.

End-to-end semantic communication paradigms demonstrate substantial potential in reducing network load and compressing data redundancy. However, their inherent openness introduces significant security risks, such as unauthorized access that enables attackers to camouflage themselves among legitimate users. Moreover, legitimate users may exploit input-output data pairs to conduct model stealing attacks. Existing defense strategies generally lack user access control mechanisms and fail to provide targeted countermeasures against model inversion attacks from internal users. To address this gap, we propose KGSC-SAT, a Key-Gated Semantic Communication framework enhanced by Steganography Adversarial Training for Secure Transmission. The framework employs a key-based feature modulation method to identify authorized users, while adversarial steganography training facilitates deep feature-level masking. Experimental results demonstrate that KGSC-SAT effectively mitigates both unauthorized access and insider model inversion threats, while delivering reliable communication performance.

Multimodal Large Language Models (MLLMs), while powerful, pose significant privacy risks by memorizing and potentially exposing sensitive information linked to individuals' visual appearances. Existing machine unlearning techniques, developed primarily for text-based models, are ill-equipped to handle the deeply entangled nature of visual and semantic knowledge. To address this challenge, we introduce GAIA-UL, a novel three-stage framework that performs Surgical Unlearning of visual knowledge. Our approach first conducts a Causal Hotspot Diagnosis, using gradient-based analysis to precisely identify influential parameters within the visual-semantic pathway. Second, it performs a Targeted Adapter Intervention, surgically injecting lightweight, trainable adapters only at these hotspots while freezing the base model. Finally, it employs Semantically Orthogonal Fine-tuning, a novel objective that forces the model's internal representation of a target face to become orthogonal to embeddings of associated sensitive concepts, thereby erasing the link at a deep representational level. Extensive experiments on the MLLMU-Bench benchmark demonstrate that GAIA-UL significantly outperforms existing baselines, achieving superior visual knowledge ablation while robustly preserving general model utility and text-only knowledge.

Sign Language recognition is practically important in various scenarios such as smart home, medical rehabilitation, and intelligent industry. Compared with wireless sensing and computer vision methods, data glove-based methods have gained a plenty of attention, because they can perform well even in the environments with multi-path noise or visual occlusion. However, existing data glove-based methods usually require complex calibration and laborious dataset collection, and suffer from accumulated error. To address these challenges, we introduce a robust sign language recognition system with cross-domain generation, called SmartGlove, the first approach to achieve robust sign language recognition. To avoid complex calibration process, we propose a customized feature set that can enable user-insensitive and unintentional system calibration. To avoid the labor cost in training data collection, we propose a cross-domain data transformation technique to generate training data in target domain. To eliminate the accumulated error of sentence recognition, we utilize a context-based calibration method considering correlation among adjacent words. We implement SmartGlove with COTS devices, and extensive experiments reveal that SmartGlove achieves accuracy exceeding 97.11% for 30 sign language words, with an average recognition time of 47 milliseconds per word. Furthermore, the system recognizes 30 common sign language sentences with accuracy of 97.17%.

Automated manufacturing is the cornerstone of the Industrial Internet of Things (IIoT) ecosystem, where vibration monitoring technology is a critical tool for maintaining industrial machinery. The prevailing approach mostly employs inertial measurement units (IMUs), lasers, and cameras, each demonstrating deployment constraints. In recent years, millimeter-wave (mmWave) radar has shown high vibration measurement performance, but it faces challenges in accurately localizing vibrating objects and determining observation points. This study introduces a new system called VibCamera, which leverages the mmWave vibration measurement technology with computer vision (CV) algorithms for vibration monitoring. With the positional assistant of CV semantic segmentation, the radar can accurately determine sufficient observation points, thereby achieving precise measurement with high directionality. VibCamera includes two camera modes, RGB-only and RGB+depth, and solves two technical challenges: 1) integrating multimodal information for vibration target localization and 2) extracting high-quality vibration signals in interference environments. VibCamera provides more consistent and precise outcomes without the need for physical contact. The experimental results indicate that the RGB-only mode has amplitude and frequency errors below$27.04 \; \mu \rm m$and 0.22 Hz, respectively, with a 90% probability, and the RGB+depth mode has errors below$23.72 \; \mu \rm m$and 0.21 Hz.

The Wi-Fi sensing technique holds great promise for future smart homes, thanks to the widespread use of Wi-Fi devices. With this technique, we can deduce the behavior of the target based on the channel state information (CSI), which is obtained during Wi-Fi communication. However, existing Wi-Fi sensing technologies are not compatible with standard communication technologies. This is because Wi-Fi sensing usually relies on capturing CSI from high-frequency communication packets, whereas regular IoT communication does not consistently maintain such high communication rates. To achieve precise sensing even with a low packet rate, we introduce LowDetrack, an indoor human detection and tracking system at ultra-low packet rates with Wi-Fi. In particular, we utilize compressed sensing to supplement missing data compared to existing systems that rely on linear interpolation or neural networks. To detect and track the target, our insights are twofold: 1) We combine compressed sensing and Fresnel zone to a theoretical model for accurately obtaining the reflection path change rate, which can be converted into the actual velocity of the target; 2) We investigate the mapping relationship between the dynamic frequency composition ratios in different links, which can provide navigation for velocity direction and correct direction recognition errors. We implement LowDetrack on commercial off-the-shelf Wi-Fi and realize human detection and tracking, where the median tracking error is 0.76m at the packet rate of 25 Hz.

Given privacy and network load concerns, employing on-device multimodal neural networks (MNNs) for IoT data is a growing trend. However, the high computational demands of MNNs clash with limited on-device resources. MNNs involve input and model redundancies during inference, wasting resources to process redundant input components and run excess model parameters. Model Redundancy Elimination (MRE) reduces redundant parameters but cannot bypass inference for unnecessary input components. Input Redundancy Elimination (IRE) skips inference for redundant input components but cannot reduce computation for the remaining parts. MRE and IRE independently fail to meet the diverse computational needs of multimodal inference. To address these issues, we aim to combine the advantages of MRE and IRE to achieve a more efficient inference. We propose anadaptivemultilevelredundancyelimination framework (AMRE), which supports both IRE and MRE.AMREfirst establishes a collaborative inference mechanism for IRE and MRE. We then propose a multifunctional, lightweight policy model that adaptively controls the inference logic for each instance. Moreover, a three-stage training method is proposed to ensure the performance of collaborative inference inAMRE. We validateAMREin three scenarios, achieving up to 52.91% lower latency, 56.79% lower energy cost, and a slight accuracy gain compared to state-of-the-art baselines.

Non-intrusive hand tracking with mmWave radar technology is important in various Human-Computer Interaction (HCI) scenarios. However, existing mmWave-based solutions require users to be stationary and restrict a fixed hand motion area, which limits application flexibility and user experience. This paper proposes a novel mmWave-basedMobileHandTracking (MHTrack) system, which tracks user's hand gestures during walking. MHTrack focuses on tracking bothabsolutehand trajectory in the global coordinate system andrelativehand trajectory to the body. Specifically, we propose a wake-up mechanism for hand motion capture, in which hand point cloud can be recognized even under body interference and noise. We propose a hand tracking strategy named local spatial update, which overcomes the sparsity and instability of point clouds, to obtain absolute hand trajectory. Subsequently, we propose a hand anchor correction method to suppress anchor offset and remove the impact of body movement from absolute hand trajectory, thereby obtaining relative hand trajectory. As a case study, we project the relative hand trajectory onto a 2D image and feed it into a gesture recognition model to recognize the gestures. We conduct extensive experiments to evaluate the performance of MHTrack. Results demonstrate a 3D hand trajectory tracking error of$3.6cm$in an area of$3.2m\times 4.8m$and a gesture recognition accuracy of$99\%$with 30 gesture classes.

Liquid sensing in ubiquitous contexts plays an essential role in various scenarios. Recently, some wireless sensing systems have been proposed for liquid identification. However, existing works usually require specific equipment or capture the signals penetrating a target, limiting the deployability of liquid sensing. In large-scale scenarios, multiple devices are usually required to expand the coverage area due to the RFID reader antenna's reading range limitation. To enlarge the sensing range and make the liquid sensing method can be adopted in real moving scenarios, in this paper, we presentMobileLiquidIDentification (MLiquID), a liquid sensing system that can recognize the type of liquid in a mobile manner with commercial off-the-shelf (COTS) RFID devices. This mobile process leads to continuous variation in location, so the major challenge in this paper is how to extract signal features from the superimposed information of movement and material. The key insight is to regard movement as an opportunity to acquire data from different perspectives instead of a challenge to hinder feature extraction. We construct a Phase-RSS model by analyzing the influence of moving and liquid on the phase and RSS signals. First, we propose a method to calculate the distance from the tag to the reader antenna. Second, we explore an identification method to identify liquid type by extracting signal features Phase-RSS coefficient$C_{P-R}$and Maximum Response Distance (MRD). Experimental results demonstrate an average accuracy of 96.80% in identifying 10 common liquids, which shows the great potential of MLiquID for mobile liquid sensing.

Location-based services and identification hold promise for future smart home applications. Through them, we can provide customized services for specific users in current locations. Recent studies have demonstrated that Wi-Fi signals can be leveraged to achieve device-free tracking and gait recognition. Despite their good performance, these two technologies are not effectively integrated for the following reasons: First, the device-free tracking method might yield tracking results that conflict with human gait. Second, extracting gait features relies on knowing or accurately estimating the user's trajectory. Consequently, gait recognition and tracking are inherently linked, but there has been no effective approach to integrate these two techniques. In this paper, we present STAGR, a system capable ofSimultaneousTrackingAndGaitRecognition. The main contribution of our technique is that we establish a theoretical model that reveals how to transform path-dependent spectra into path-independent spectra directly. Specifically, we conduct a preliminary study to demonstrate the need for simultaneous tracking and gait recognition. Second, we propose a novel method to extract path-independent gait features, which can significantly save execution time compared with the learning-based method. Third, we design a polar-coordinate filtering method to retain the gait features while correcting the trajectory. We implement a prototype STAGR system and conduct extensive experiments to verify the proposed mechanism. The experimental results show that we can realize simultaneous tracking and gait recognition. The median tracking error is$ 0.45m$, while the recognition accuracy is 95.3% for 6 users.

Wi-Fi contact-free sensing systems have attracted widespread attention due to their ubiquity and convenience. The integrated sensing and communication (ISAC) technology utilizes off-the-shelf Wi-Fi communication signals for sensing, which further promotes the deployment of intelligent sensing applications. However, current Wi-Fi sensing systems often require prolonged and unnecessary communication between transceivers, and brief communication interruptions will lead to significant performance degradation. This paper proposes Baton, the first system capable of accurately tracking targets even under severe Wi-Fi feature deficiencies. To be specific, we explore the relevance of the Wi-Fi feature matrix from both horizontal and vertical dimensions. The horizontal dimension reveals feature correlation across different Wi-Fi links, while the vertical dimension reveals feature correlation among different time slots. Based on the above principle, we propose the Simultaneous Tracking And Predicting (STAP) algorithm, which enables the seamless transfer of Wi-Fi features over time and across different links, akin to passing a baton. We implement the system on commercial devices, and the experimental results show that our system outperforms existing solutions with a median tracking error of 0.46m, even when the communication duty cycle is as low as 20.00%. Compared with the state-of-the-art, our system reduces the tracking error by 79.19% in scenarios with severe Wi-Fi feature deficiencies.

Federated learning (FL) is a distributed learning paradigm that enables multiple client devices to collaboratively train a global model based on their local datasets while protecting data privacy. However, due to its distributed nature, FL is susceptible to the resources of heterogeneous client devices with different data quantities, communication resources, and computing capabilities. Heterogeneity leads to uncertain global model training time and hinders the convergence of the global model. Therefore, selecting suitable clients to participate in the FL training process is necessary to improve the efficiency of FL. This paper proposes an FL client selection method (FedAHP) based on the Analytic Hierarchy Process (AHP) to optimally balance the trade-off between model accuracy and training time during client selection. Experiments show that FedAHP outperforms the greedy method regarding training time consumption and model accuracy. Specifically, FedAHP achieves a 67% reduction in communication rounds compared to the greedy method when the model accuracy reaches 0.90. Furthermore, when taking the same 20 hours, FedAHP improves the accuracy of the global model by 6% in comparison to the greedy method.

With the wide application of multi-modal fusion sensing in scenarios such as autonomous driving and human-computer interaction, the privacy security and communication burden caused by massive data uploading need to be solved urgently. Federated Learning (FL) has received significant attention as a privacy-preserving distributed machine learning paradigm. Recent Multi-Modal Federated Learning (MMFL) focuses on addressing modal heterogeneity to enhance accuracy and speed up convergence. However, it overlooks the huge communication overhead in updating complex multi-modal network models, especially in edge environments with limited bandwidth. At the same time, the state-of-the-art communication-efficient FL methods are not customized to the MMFL characteristics. In this paper, we propose the Adaptive Quantization framework for Multi-modal Federated Learning (AQMFL). AQMFL implements decision-level multi-modal fusion locally by using parallel training and model ensemble, supporting its adaptation to modal heterogeneity and flexible deployment. AQMFL can adaptively allocate the number of quantization levels of gradient according to the modal contribution and the heterogeneous communication ability of nodes, which speeds up the system convergence and achieves a better balance between accuracy and communication efficiency. Compared with the classical baselines, AQMFL can reduce the total communication overhead by up to 50.47% and the total training time by up to 52.11% while maintaining the accuracy.

The ubiquity of acoustic devices and the fine-grained sensing of acoustic signals have made acoustic device tracking a popular option. We propose to expand the use of commercial devices with microphones as an extension of the audio system to support intelligent applications, such as VR/AR. This paper introduces a novel 6D acoustic pose estimation system. To realize device-based pose estimation, most existing systems deploy multiple speakers. However, due to limited inaudible bandwidth, concurrent transmissions with multiple speakers pose challenges in balancing resolution and frame rate. To address this problem, we design 2×Track, a band multiplexing signal model that doubles the availability of limited bandwidth by utilizing a unique encoding strategy for concurrent transmissions. We also propose solutions to enhance signal feature estimation and implement a 6DoF pose tracking scheme tailored for distributed systems. The prototype is deployed on a typical circular microphone array, and experimental results show that 2×Track achieves a median position and orientation error of 7.6mm and 4.1°, respectively, in a 4-speaker setup. Our extended applications on commercial devices also showcase the versatility of our system, particularly in face orientation detection, air mouse and drone tracking.

Voice command systems have been widely deployed on many smart devices for remote control. To further enrich the intelligence of these smart devices, the location of sound plays an important role in context-aware acoustic services. Despite initial steps made toward reliable voice localization, the state of the arts rely on prior knowledge of device location, device orientation and an indoor electronic map. To mitigate this additional cost, this paper presents VoiceMap, an autonomous mapping system of acoustic devices for voice localization. The insight behind VoiceMap is to explore the cooperation of sweeping robots and voice devices. Specifically, the sweeping robot is responsible for exploring the electronic map of the environment, while the microphone array is responsible for localizing the sweeping robot, so that we can establish the positional relationship between them. The core challenges are how to accurately locate the continuously moving robot, and how to synchronize the coordinate systems of the sweeping robot and the voice devices. To this end, we first design an inertial-based super-resolution method to estimate the angle of arrival (AoA) with respect to the robot. Then, we develop an effective coordinate synchronization mechanism, so that VoiceMap can automatically locate the voice devices on the electronic map generated by the robot. Finally, we implement a prototype system using commercial devices, and conduct comprehensive experiments to verify the proposed system. The experimental results show that we can realize a median error of 0.12m in terms of device localization.

Wi-Fi-based noncontact respiratory monitoring technology plays a significant role in smart elderly care by eliminating the need for users to wear additional hardware devices. Although respiration sensing has shown impressive accuracy in ideal scenarios, achieving precise respiration monitoring in interfering scenarios, such as when other users, are engaged in activities, remains challenging. This difficulty arises primarily from the weak nature of the respiratory signal, which is susceptible to interference from activities. To address this issue, this article introduces the RespEnh system, a respiratory monitoring system that utilizes human location information to mitigate interference. First, We enhance the quality of respiratory signal by leveraging the frequency diversity of Wi-Fi signal, effectively visualizing respiratory patterns even amidst interference. Second, we apply short time window shift to remove walking noise in the time domain while preserving respiratory signal characteristics. Finally, we integrate multiantenna signals to enhance spatial domain performance. Experimental results demonstrate that our system achieves a relative anti-interference performance of 2.4 m, even when breathing as far away as 6.4 m. It effectively monitors person’s respiration, with a detection rate of 80% within an error range of 1 bpm.

Sleep posture recognition is practically important in various scenarios such as sleep healthcare, bedridden patient care, and chronic disease diagnosis. With concerns of user privacy preserving, we prefer the wireless sensing methods to computer vision methods when dealing with sleep posture recognition. However, the existing wireless sensing methods suffer from at least one of the following major limitations: (i) difficult to deploy in practice; (ii) few posture categories; (iii) insufficient accuracy; (iv) poor generalization ability. In this paper, we use commercial-off-the-shelf (COTS) mmWave radar to implement a sleep posture recognition system called PosMonitor. When designing the PosMonitor system, we need to address the following challenging issues. First, we propose an angle purification method based on multi-frame joint analysis to alleviate the sparsity and instability of the point cloud. Then, we endow the point cloud with respiratory features to enhance its representation of the sleep posture. Further, to make the system applicable to different users, we extract relative respiratory features by normalization to overcome individual differences. Extensive experimental results show that our PosMonitor system can achieve 98% accuracy on average in recognizing 6 typical sleep postures and has good reliability across different conditions.

Wireless signal analytics in IoT systems can enable various promising wireless sensing applications such as localization, anomaly detection, and human activity recognition. As a matter of fact, there are significant correlations in terms of dimension, spatial and temporal aspects among wireless signals from multiple sensors. However, none of the wireless sensing research currently in use directly incorporates or exploits the signal correlations. Therefore, there is still substantial scope for improvement in regards to accuracy and robustness. We are introducing a novel framework called Signal Correlation Learning (SCL). This framework utilizes a directed graph to explicitly represent the signal correlation across various wireless sensors. We use signal embedding to depict the correlation features of a multi-dimensional sensor that arise from a multi-sensor system. Then, we perform Kullback-Leibler (KL) divergence on embedding vectors of any pair of sensors in the system to construct a subgraph at a given time point, which can measure the spatial signal correlation of sensors. Subsequently, several subgraphs spanning a specific time frame are fused into a coherent universal graph based on the small-world theory. This universal graph represents the three types of signal correlation simultaneously. A signal correlation aggregation structure is utilized to extract the features from the universal graph. These features can be used to address target sensing problems. We implement SCL in real RFID, Bluetooth, WIFI, and Zigbee systems, and evaluate its performance in three common wireless sensing problems including localization, anomaly detection, and human activity recognition. Extensive experiments demonstrate that our SCL framework significantly outperforms state-of-the-art wireless sensing algorithms by increasing$80\%\sim 190\%$in terms of accuracy, and by increasing$160\%\sim 220\%$in terms of robustness.

The ubiquitous Internet-of-Things (IoT) devices generate vast amounts of multimodal data, and the deep multimodal fusion network (DMFN) is a promising technology for processing multimodal data. Deploying DMFNs locally on embedded IoT devices is a profitable way to provide privacy-preserving and robust sensing services. However, the current compression methods suffer from the following limitations: First, they are designed based on unimodal networks or specific model structures. Hence, it is hard to extend these methods to diverse DMFNs; Second, existing works never relate their efforts to disparate computational demands of multimodal data and modalities. Easy samples and redundant modalities consume the same computational resources as powerful modalities and complex samples. We propose anAdaptiveCompressionFramework (ACF) for DMFNs to address those challenges. It enables input-dependent runtime compression locally on resource-constrained embedded devices. Specifically, we propose an offline model transformation module to upgrade the static network with two kinds of dynamic components to support online structural adjustment. Then we design a lightweight policy network to generate multi-granularity and data-dependent compression strategies for different model parts. Finally, we evaluate ACF on four DMFNs across three embedded platforms. Compared with the best results of the existing schemes, ACF obtains up to 2.61× latency reduction and 2.30× energy consumption reduction, with up to 3.57% accuracy improvement.

Fine-grained recognition of human manipulation activities on objects is crucial in the era of human-computer-object integration. However, there is a lack of solutions for simultaneous recognition of human identity, manipulation activities (including drawing and rotation), and manipulated objects. Therefore, we propose an RF-Camera system that combines RFID and computer vision techniques to address this challenge in multi-person and multi-object scenarios. In RF-Camera, we employ a skeleton-assisted method to extract facial images of target individuals, enabling precise recognition of their identities. To identify manipulation activities, we analyze the 3D hand trajectory and fingertip vector angle, differentiating drawing and rotation manipulation activities. Additionally, we model target person?s hand movements to predict phase data of the target tag, enabling the determination of person-object relationships. Implementing RF-Camera using COTS RFID and Kinect devices involves overcoming challenges such as extracting effective data from noisy streams, predicting virtual phase data considering hand-tag offset, and ensuring high tag reading rates in tag-dense scenarios. We conducted experiments involving six participants performing object manipulation activities, including drawing letters/symbols and rotating movements. Extensive experimental results show that RF-Camera achieves over 90% accuracy in recognizing person identity, manipulation activities, and person-object matching in most conditions.

The smart speakers benefit from the ability to localize and identify users. Specifically, we can analyze the user's habits from the historical trajectory to provide better voice-based services. However, current voice localization method requires the user to actively issue voice commands, which makes smart speakers unable to track and identify silent users most of the time. This paper introducesWSTrack+, a system that combinesWi-Fi andSound to track human movement and recognize gait patterns. In particular, current smart speakers naturally support both Wi-Fi and acoustic functions. As a result, we are able to construct the system using just one router and a smart speaker, which is a more promising approach compared to existing systems that rely on multiple routers for sensing. To track and identify the silent user, our insights are twofold: 1) the smart speakers can hear the sound of the user's footstep, and then extract which direction the user is in; 2) we can extract the reflected path change rate from the Wi-Fi signals, and the acoustic signal can help us convert the path change rate into the actual user's velocity. Our implementation and evaluation on commodity devices demonstrate thatWSTrack+can realize simultaneous tracking and gait recognition, where the median tracking error is$0.34m$and the recognition accuracy is 88.6% for 12 users.

The evolution of Wi-Fi to next-generation 802.11bf demonstrates the potential of device-free Wi-Fi sensing applications, where we can remotely infer the behaviors of users without bringing into physical contact with them. Among these sensing applications, Wi-Fi tracking is critical to provide location based services. Recent Wi-Fi tracking systems can be cataloged into model-based and data-based approaches: (1) the model-based approach is to build the mathematical tracking model. However, this method is sensitive to environmental noise, and spends more execution time; (2) the data-based approach is to train a neural network. However, this method requires a lot of efforts to collect training dataset, and cannot handle all types of trajectories well. To resolve these issues, we propose theNNE-Tracking, a Neural Network Enhanced tracking framework. The core design principle ofNNE-Trackingis as follows: we improve the tracking accuracy based on the data-based approach, and utilize the model-based approach to supervise whether the neural network is already working well. Moreover, we also design a framework to estimate unknown parameters of the tracking model, so that the system can automatically generate the Wi-Fi map. We take the Wi-Fi passive tracking as a specific example to explain how to applyNNE-Trackingin practical applications. Experimental results demonstrate that our design can reduce 59.4% ∼ 85.3% tracking errors while significantly saving execution time. As for deployment costs, we can automatically infer the Wi-Fi map without manual calibration; As for stability, when we repeat the training process with different hidden layers and random seeds, the tracking standard deviation of these neural networks is only 1.4cm.

This paper presents MobiScatter, which enhances capacity of CSS based backscatter networks for accommodating drone-carried access points (APs). CSS based backscatter design has favorable features including long range and high concurrency. However, the concurrency of the network can be reduced by 34% when the drone-carried AP is introduced due to the mobility and fast fading. In order to maintain high concurrency, MobiScatter presents a series of new designs. In particular, we propose to enhance concurrency of the CSS based backscatter network with symmetric upchirps and downchirps, which neutralizes the impact of imperfect frequency orthogonality. Then, to mitigate the impact of fast fading on decoding, we present a novel half-period chirp modulation scheme for crossed chirps. Finally, we provide a power management method for tags by controlling transmitting time of chirps. We construct a MobiScatter prototype, which contains a drone-carried AP implemented with a mobile USRP and 200 tags. We deploy those tags in an area of$200 m\times 180 m$on a meadow. Experimental results show that MobiScatter can support 160 concurrent backscatter transmissions when the AP moves at$15 m/s$.

A wide range of scenarios, such as warehousing, and smart manufacturing, have used RFID mobile robots for the localization of tagged objects. The state-of-the-art RFID-robot based localization works are based on the premise of stable speed. However, in reality this assumption can hardly be guaranteed because Commercial-Off-The-Shelf (COTS) robots typically have inconsistent moving speeds, and a small speed inconsistency will cause a large localization error. To this end, we propose a Speed Inconsistency-Immune approach to mobile RFID robot Localization (SILoc) system, which accurately locates targets when the robot moving speed varies or is even unknown. We propose an optimized unwrapping method to maximize the use of data, and a lightweight algorithm to calculate the locations in both 2D and 3D spaces. By utilizing the characteristics of tag-antenna distance and combining the phase data from multiple antennas, SILoc can effectively eliminate the side effects of speed inconsistency. To increase the flexibility, we further optimize the system and propose SILoc$+$, which enables the system to achieve localization with part of the data, keeping speed inconsistency-immune. Extensive experiments demonstrate that SILoc and SILoc$+$can achieve a centimeter-level localization accuracy in the scenario with an inconsistent or unknown robot moving speed.

As a diagnostic method of equipment operational status, vibration monitoring plays a significant role in industrial systems. It is necessary to monitor multiple equipment components simultaneously, due to their different vibration modes. Previous solutions either work in an invasive manner or face challenges in object localization and results correspondence. Therefore, we propose VibCamera, a vibration monitoring system that combines mmWave radar and computer vision technology. We propose an expand-shrink method to optimize object detection results of computer vision and combine camera localization results to extract mmWave signals. Additionally, we employ mmWave data recombination and respective fitting methods to calculate the vibration characteristics for each point accurately. The experiment shows that after fusing visual information, the target detection accuracy is improved to 94.8%, and the cluster point efficiency is improved by 23.3%. Furthermore, amplitude and frequency measurement errors are reduced to 29.1μm and 0.08Hz, respectively.

Device-free Wi-Fi tracking has become essential for ubiquitous wireless sensing. However, current device-free Wi-Fi tracking systems suffer from two limitations: First, abnormal signals interfere with tracking performance when the user walks across the direct link of the transceivers and second, the tracking error based on the velocity integral accumulates over time. This article proposes CrossTrack, the first device-free cross-link tracking system with commodity Wi-Fi. Our inspiration is to regard the cross-link behavior as an opportunity to correct the trajectory instead of disturbing noise like previous work. Our approach involves three main steps. First, we devise a metric that is capable of detecting cross-link behavior. Second, we propose a new theoretical model that identifies the cross-link position as a landmark. Third, we develop a path revision technique that utilizes this landmark to optimize the trajectory. The technique innovation of this article is to reveal the theoretical approach to transform cross-link interference into optimization in device-free tracking for the first time. We implement CrossTrack based on commercial Wi-Fi devices and conduct comprehensive experiments. Our results show that CrossTrack can reduce tracking errors by 48.75%, and the median tracking error is 0.41 m.

Wi-Fi CSI-based indoor localization systems can realize decimeter-level localization accuracy. However, these systems require that the location and antenna array orientation of Wi-Fi Access Point (AP) are known in advance, which makes it impractical for large-scale deployment. In this paper, we present MapFi, which can realize autonomous mapping of Wi-Fi infrastructure without labor-intensive site survey. To this end, we focus on addressing three problems. First, as there will be diverse layouts of devices and antennas with respective to numerous and heterogeneous Wi-Fi APs, we propose a general method to estimate AoA and generate the Wi-Fi map. Second, while the existing systems can provide a promising median localization accuracy, tail performance is usually far worse. Consequently, we develop a revision method to reduce tail errors. Third, when deployed in large-scale indoor environment, obstacles and long-distance communication might incur failed CSI collection. Therefore, we segment Wi-Fi APs into groups and finally merge these groups to generate the global Wi-Fi map. We conduct experiments in different scenarios to verify the proposed methods. The experimental results show that we can realize the$80\%$localization error within$1.15m$and$0.74m$in office room and open space respectively, which is as accurate as localization systems requiring known Wi-Fi map.

This paper studies the problem of 6D pose estimation, which is practically important in various application scenarios such as robotic-based object grasping, obstacle avoidance in autonomous driving scene, and object integration in mixed reality. However, existing methods suffer from at least one of the five major limitations: dependence on object identification, complex deployment, difficulty in data collection, low accuracy, and incomplete estimation. To overcome the above limitations, this paper proposes an RC6D system, which is the first to estimate 6D poses by fusing RFID and Computer Vision (CV) data with multi-modal deep learning techniques. In RC6D, we first detect 2D keypoints through a deep learning approach. We then propose a novel RFID-CV fusion neural network to predict the depth of the scene, and use the estimated depth information to expand the 2D keypoints to 3D keypoints. Finally, we model the coordinate correspondences between the detected 2D-3D keypoints, which is applied to estimate the 6D pose of the target object. When implementing RC6D, we mainly address the following three technical challenges. (i) To predict 6D poses without using the CAD model, we propose a network architecture for monocular depth estimation. (ii) To train the neural network for 6D pose estimation without time-consuming 6D labeling, we use an unsupervised learning algorithm based on 2D-3D point pair matching. (iii) To detect the subject of the object without identification, we leverage optical flow to restrict the object and RFID to directly obtain its information. The experimental results show that the localization error of RC6D is less than 10 cm with a probability higher than 90.64% and its orientation estimation error is less than 10° with a probability higher than 79.63%. Hence, the proposed RC6D system performs much better than the state-of-the-art related solutions.

With the wide deployment of RFID in various scenarios such as warehouse management, freight transportation, and manufacturing, tag authentication is increasingly important due to the threat of counterfeit tags. Recent physical-layer authentication approaches have demonstrated that the subtle differences in the hardware features offer a unique fingerprint to authenticate a tag. Although the state-of-the-art approaches are effective in laboratory environments, they are difficult for practical deployment because they either require complex analysis of the raw signal propagation or restrict the geometrical positions of the tags. In this paper, we propose an RFID tag authentication based on frequency- and orientation-related phase fingerprints, called FopPrint, which does not require raw signal analysis or complex geometric relationship. FopPrint uses the phase values of tags at different frequencies and orientations to construct feature matrices as physical-layer fingerprints and uses a pair of adjacent tags as identifiers of each object. FopPrint can effectively eliminate the influence of environmental factors by using the feature matrix constructed by the phase difference. We implement a prototype of FopPrint using Commercial-Off-The-Shelf (COTS) RFID devices. Extensive experimental results show that FopPrint achieves high authentication accuracy of 94% in various experimental settings.

Eye tracking is playing an important role in many applications including human-computer interaction and behavior study. However, the existing approaches have at least one of the following limitations: (i) dedicated devices such as infrared camera and eye-tracker are required; (ii) complex calibration process is involved; (iii) substantial computing resources are consumed; (iv) users suffer from the risk of privacy leakage. To address the above limitations, we propose a H eatmap-based E ye T racking (HETrack) system. One of the key challenges in our system is to design a lightweight model for fine-grained tracking when the computing resources of device is limited. Also, it is necessary to protect user privacy in such a system. To address the above challenging issues, the proposed system consists of the following processes. First, when users randomly look at the screen of the device, HETrack obtains the raw image containing facial information. Then, we design a neural network model and train it with federated learning. The model can map the image to heatmap that implies the possibility of the user’s gaze position on the screen. Finally, HETrack can intercept the real-time video stream into frames, and employ the trained model to generate the heatmap of current frame for gaze estimation. We implement HETrack based on a Commercial-Off-The-Shelf (COTS) camera and conduct extensive experiments to evaluate its performance. Our HETrack system only requires once calibration; whereas, the state-of-the-art work proposed by Google requires 3~5 times calibration on average. Unlike previous approaches that transmit raw image data to a central server, in our HETrack system, only parameters are transmitted, thereby well protecting the user’s privacy. Experimental results demonstrate that the average distance error of estimated gaze point is 3cm, which is compatible with the state-of-the-art methods.

Mobile RFID robots have been increasingly used in warehousing and intelligent manufacturing scenarios to pinpoint the locations of tagged objects. The accuracy of state-of-the-art RFID robot localization systems depends much on the stability of robot moving speed. However, in reality this assumption can hardly be guaranteed because a Commercial-Off-The-Shelf (COTS) robot typically has an inconsistent moving speed, and a small speed inconsistency will cause a large localization error. To this end, we propose a Speed Inconsistency-Immune approach to mobile RFID robot Localization (SILoc) system, which can accurately locate RFID tagged targets when the robot moving speed varies or is even unknown. SILoc employs multiple antennas fixed on the mobile robot to collect the phase data of target tags. We propose an optimized unwrapping method to maximize the use of the phase data, and a lightweight algorithm to calculate the locations in both 2D and 3D spaces based on the unwrapped phase profile. By utilizing the characteristics of tag-antenna distance and combining the phase data from multiple antennas, SILoc can effectively eliminate the side effects of moving speed inconsistency. Extensive experimental results demonstrate that SILoc can achieve a centimeter-level localization accuracy in the scenario with an inconsistent or unknown robot moving speed.

Channel state information (CSI) based Wi-Fi localization can achieve admirable decimeter-level accuracy; however, such systems require labor-intensive site survey to calibrate the AP position and the antenna array direction, which hinders practical large-scale deployment. In this article, we reveal an interesting finding that the calibration efforts for deploying the CSI localization system can be significantly reduced by simply replacing the ordinary linear antenna layout of the AP with the non-linear layout. In particular, we first present an autonomous self-calibrating method to significantly facilitate site survey for deploying CSI localization systems. Then we propose a systematical evaluation mechanism to show the fundamental reason why linear antenna layout usually leads to serious errors and why non-linear antenna layout is better off. Finally, we build a testbed with COTS devices and conduct comprehensive experiments. Results show that triangular antenna layout can achieve 80% angle of arrival (AoA) measurement error within 9° for any direction in contrast to 16° based on linear antenna layout. Moreover, we can realize promising localization accuracy as previous works even without labor-intensive site survey, where 80% localization error is within$0.60m$.

Fingerprinting indoor localization systems exploit wireless signal propagation features to estimate the location of wireless devices, where the major challenge in practice is the all-consuming training process: it requires site survey to establish the mapping between the signal feature and the location where the feature is observed. In this paper, we present a Wi-Fi localization scheme based on channel state information (CSI) of wireless signals, which manages to relieve time-consuming site survey. In particular, we first propose how to automatically generate the theoretical fingerprints database based on the signal propagation model and geometric methods. Localization with the theoretical fingerprints database yields accuracy close to existing methods. Second, we improve localization accuracy by parsing the user's trajectory instead of restricting to the single spot, where human movement features introduce more information for localization. Third, we present an automatic update scheme for the theoretical fingerprints database to improve time efficiency for localization, which can save 94 - 98% processing time for utilizing the CSI fingerprints database. We implement a prototype with COTS devices and conduct comprehensive experiments to verify proposed mechanisms. Results show that our design achieves 80% localization errors within 0.3m, which is 3× accuracy compared with the state-of-the-art design leveraging CSI.

In this paper, we for the first time demonstrate how to realize multi-frequency backscatter communication for low power IoT devices. Our key innovation is an encoding rule that works from O(100) MHz to 2.4 GHz to cover most commercial communication bands such as BLE, LoRa and Wi-Fi. Based on the proposed method, backscatter devices can adaptively select one appropriate wireless protocol according to application scenarios. To this end, we reveal the fundamental reason why the existing backscatter tags only work in a specific frequency range and propose a universal multi-frequency communication method. We build hardware prototype and conduct experiments at several frequencies from 150 MHz to 2.4 GHz. Experimental results show that UniTag significantly improves communication performance at different frequencies in contrast to the existing methods, where the average bit error rate is only 13% to 50% of existing methods.

Recent studies show that the voice control system (VCS) is subject to the inaudible voice command attack, which can not be heard by human ears but can be recorded by the microphone. An adversary could leverage the attack to disable the VCS user's home security system, leak the victim's privacy or download malware stealthily. Efforts have been dedicated to developing forensics based defense mechanisms, which target at detecting traces of the attack signal; however, we find that existing approaches of the kind still leave loopholes. Moreover, a complete defense mechanism should be able to not only detect the attack but also cancel out the attack signal, and meanwhile ensure the legitimate voice commands unaffected, which however is still unavailable to the best of our knowledge. This paper is an attempt to fill the gap. We first systematically analyze existing forensics based defense mechanisms and reveal the root cause of their loopholes. Then we present an active inaudible-voice-command cancellation (AIC) design, which can reliably detect and capture the attack signal facilitated by our custom-designed "guard'' signal transmitter. AIC can create a special spectrum in the passband of the VCS microphone, based on which we are able to neutralize the attack signal in software means. We implement a prototype of our defense system and conduct comprehensive experiments to validate our design.

Channel state information (CSI) based Wi-Fi localization can achieve admirable decimeter-level accuracy; however, such systems require labor-intensive site survey to calibrate the AP position and the antenna array orientation, which hinders practical large-scale deployment. In this paper, we reveal an interesting finding that the calibration efforts for deploying the CSI localization system can be significantly reduced by simply replacing the ordinary linear antenna layout of the AP with the non-linear layout. In particular, we first present an autonomous self-calibrating method to significantly facilitate site survey for deploying CSI localization systems. Then we propose a systematical evaluation mechanism to show the fundamental reason why linear antenna layout usually leads to serious errors and why non-linear antenna layout is better off. Finally, we build a testbed with COTS devices and conduct comprehensive experiments. Results show that triangular antenna layout can achieve 80% angle of arrival (AoA) measurement error within 9◦for any direction in contrast to 16◦based on linear antenna layout. Moreover, we can realize promising localization accuracy as previous works even without labor-intensive site survey, where 80% localization error is within 0.60m.

Self-calibrating wireless indoor localization systems construct the radio map even the indoor floor plan automatically, which avoids the labor-intensive site survey process; however, existing systems utilizing the feature of Wi-Fi signals can only provide coarse-grained indoor maps, which hinders improvement of localization accuracy. In this paper, we present FineLoc, a fine-grained self-calibrating localization system based on the freely-deployed Bluetooth low energy (BLE) nodes and crowd-sourced data, which can profile more detailed layout information of the indoor space. We first reveal that existing systems can only generate inaccurate floor plans owning to the coarse-grained Wi-Fi reference information. Then, we utilize the increasingly popular BLE beacon nodes as the source of reference information, with which a series of dead-reckoning optimization and new schemes particularly for finer-grained indoor map construction are presented. We implement a prototype FineLoc system, which is deployed in around 11,000 m2areas. Our experimental results with the prototype show that FineLoc can achieve 80 percent localization errors within 1.6 m, 1.4 m, and 1.1 m in the library, classroom building, and office building, respectively, with an average density of deployed BLE nodes less than 2.6/100 m2.