Device-free wireless sensing (DFWS) has gained significant attention due to its high accuracy and privacy-preserving capabilities. DFWS systems work by analyzing the influence pattern of targets on the surrounding wireless signals. However, changes in the sensing scenario can alter signal propagation patterns, causing deep learning models to lose effectiveness in cross-scenario applications. To address this problem, we analyze the information content of different samples from a free-energy view, and provide a new idea to guide the alignment of target scenario samples with source scenario samples based on free-energy. We find that free-energy can measure the degree of scenario knowledge contribution of the samples. Based on this observation, we first perform unsupervised coarse alignment by minimizing the free-energy deviation between scenarios. Next, we iteratively select a few number of high-free-energy samples near the decision boundary to fine-tune the network, achieving scenario fine alignment with a small labeling effort. Extensive experiments on two public datasets and one self-collected dataset show that our proposed method achieves high accuracy in cross-scenario human activity and gesture recognition tasks.

Device-Free mmWave Sensing (DFWS) could sense target state by analyzing how target activities influence the surrounding mmWave signals. It has emerged as a promising sensing technology. However, when employing DFWS indoors, specular reflection interference arises due to the specular reflectors. This interference often induces ghost targets, impacting the accurate estimation of the number and position of targets, resulting in degradation in sensing performance. To tackle this issue, we delve into the generation mechanism of specular reflection interference and analyze its multi-domain characteristics. Through exploration, we discern its temporal sparsity, spatial symmetry or collinearity, and frequency correlation characteristics, and propose four metrics to measure them, accordingly. Specifically, we propose a temporal characteristic quantitative evaluation metric based on identity matching, spatial symmetry and collinearity quantitative evaluation metrics based on geometric analysis, and a frequency correlation quantitative evaluation metric based on Doppler velocity correction, respectively. Based on these metrics, we design a novel Specular Reflection Interference Mitigation (SRIM) method and develop a robust SRIM-DFWS prototype system based on a 60 GHz mmWave radar to validate our proposed method. Experimental results demonstrate that our proposed method could achieve accurate and effective mitigation of specular reflection interference in device-free target tracking.

Device-free positioning (DFP) using mmWave signals is an emerging technique that could track a target without attaching any devices. It conducts position estimation by analyzing the influence of targets on their surrounding mmWave signals. With the widespread utilization of mmWave signals, DFP will have many potential applications in tracking pedestrians and robots in intelligent monitoring systems. State-of-the-art DFP work has already achieved excellent positioning performance when there is one target only, but when there are multiple targets, the time-varying target state, such as entering or leaving of the wireless coverage area and close interactions, makes it challenging to track every target. To solve these problems, in this paper, we propose a spatial-temporal analysis method to robustly track multiple targets based on the high precision mmWave point cloud information. Specifically, we propose a high precision spatial imaging strategy to construct fine-grained mmWave point cloud of the targets, design a spatial-temporal point cloud clustering method to determine the target state, and then leverage a gait based identity and trajectory association scheme and a particle filter to achieve robust identity-aware tracking. Extensive evaluations on a 77 GHz mmWave testbed have been conducted to demonstrate the effectiveness and robustness of our proposed schemes.

Empowered by the feature extraction ability of deep neural networks (DNNs), the DNN-based device-free wireless sensing (DFWS) could recognize human activity by analyzing the pattern information involved in the influenced wireless signals. However, labeling samples is time-consuming and labor-intensive because wireless signals are not human-interpretable. In practical applications, there are always few labeled samples, and how to realize high-performance DFWS with few labels becomes an urgent problem to solve. To tackle this problem, finding compact representative features for samples in an unsupervised manner is crucial. To this end, we design a contrastive learning framework to obtain features of unlabeled samples by maximizing the mutual information between features and the corresponding samples. The contrastive training process extracts features for the input samples by contrasting positive and negative sample pairs, thus strengthening the correlation between the features and the corresponding samples. The intuition behind our method is that mutual information measures the correlation between features and samples, and thus the maximum mutual information could capture informative and discriminative features. Our evaluation results on two publicly available data sets and one data set collected by ourselves show that our proposed method achieves satisfactory accuracy for both human activity and gesture recognition tasks with few labels.

Device-free gesture recognition has attracted significant attention due to its potential applications in pervasive interaction. It enables gesture recognition in a device-free and contact-free manner by analyzing the influence pattern of human gestures on surrounding wireless signals, such as mmWave signals. Although remarkable progress has been achieved in this area, the recognition performance will degrade remarkably when gestures are conducted in different scenarios. In this paper, we leverage mmWave signals to design two robust trajectory features, i.e., the trajectory image and the trajectory time-sequence features, that are independent of the conducted scenarios to solve the aforementioned problems. Specifically, we employ the particle filter algorithm to construct the raw trajectory image utilizing range measurements, rotate and enhance the image to obtain the trajectory image feature suitable for recognition by leveraging a public handwriting font image data set as the training set. Additionally, we derive the range of the trajectory relative to a stable point as the trajectory time-sequence feature. With these trajectory features, we design a deep network to perform the gesture recognition task. To validate the effectiveness of the proposed methods, we conduct extensive experiments on a 77GHz mmWave testbed. The results indicate that the two proposed trajectory features are feasible for achieving scenario-independent gesture recognition.

Device-free vital signs monitoring is an emerging technology that utilizes the unique influence of chest vibrations on surrounding wireless signals to achieve vital signs monitoring in a device-free and contact-free manner. Existing methods could achieve good monitoring performance when high-quality reflected signals can be obtained. However, in daily vital signs monitoring at home, the received reflected signals are often very weak due to factors such as obstruction and attenuation, resulting in a sharp decrease in the monitoring performance. To address the aforementioned challenges, in this paper, we develop a diversity-enhanced robust device-free vital signs monitoring system using mmWave signals. Specifically, inspired by the concept of diversity in the field of communications, we propose a diversity-enhanced wireless sensing strategy that comprehensively utilizes multi-dimensional physical layer resources, including antennas, chirps, and space, to improve the signal-to-noise ratio of vital signs. Additionally, inspired by cameras that achieve clear images by prolonging exposure time, we propose an accumulation-enhanced localization method to lock onto the chest of the human body in complex scenarios. Extensive experiments on a 60 GHz mmWave testbed demonstrate that our developed system could guarantee robust vital signs monitoring performance in various challenging scenarios, even at distances of up to 40 m.

Device-free wireless sensing (DFWS) has drawn lots of attention due to its potential application in the fields of human–computer interaction and smart home. Deep networks-based DFWS technique has achieved excellent sensing performance. However, network complexity limits its deployment on resource-limited sensing devices. A feasible way is to implement a simple network to accomplish the DFWS task. However, the sensing performance will drop dramatically due to its limited learning ability. In this article, to realize lightweight DFWS with acceptable performance, we propose an information-for-complexity strategy to promote the learning ability of the simple network. We leverage knowledge distillation framework to explore external information to augment the extrinsic learning ability, and utilize multiscale receptive fields to explore the internal information to augment the intrinsic learning ability. Extensive experiments on a 77 GHz mmWave testbed show that the performance degradation of the lightweight DFWS system is within 3%, while the complexity decreases remarkably.

Device-free gesture recognition (DFGR) is an emerging technique which could leverage the influence of human gestures on surrounding wireless signals to recognize gestures. It has gained widespread attention due to its promising prospect of empowering pervasive wireless devices with the sensing ability. Due to the inconsistency of the feature distribution in different scenarios, a well-trained DFGR system often fails to get satisfactory performance in cross-scenario conditions. Researchers have done valuable exploration on alleviating the feature distribution shift from a global distribution point of view. However, global feature distribution alignment could not solve the feature distribution shift problem completely. In this article, we develop a self-adaptive adversarial learning network which could further reduce the feature distribution shift through aligning the local feature distribution. Specifically, we design an adversarial network which is consisted of a feature extractor, a scenario discriminator, and two diverse classifiers. It could evaluate the degree of local feature distribution alignment by analyzing the prediction inconsistent of the classifiers. We design a self-adaptive adversarial loss which can be adjusted adaptively according to the degree of local alignment. If the features have been aligned locally, we reduce their impact on the loss to protect these aligned features. Otherwise, we increase their influence to accelerate the training process. The extensive experiments conducted on a designed mmWave testbed demonstrate that the proposed method could achieve an accuracy of at least 4% higher than those of existing cross-scenario DFGR methods, while the number of training iterations can be reduced by nearly half.

Device-free gesture recognition is a potential noncontact human–computer interaction technique. It leverages the unique influence of the conducted gesture on surrounding wireless signals to accomplish gesture recognition. Existing methods usually leverage doppler spectrogram of the influenced wireless signals to characterize the motion pattern of gestures. These methods have achieved satisfactory accuracy when the gestures are conducted in a relatively fixed location, direction, and speed. However, when gestures are conducted in a different scenario, the recognition accuracy will drop dramatically. In this article, we try to solve this issue by characterizing the gesture motion pattern using a novel robust intrinsic spectrogram, which is independent of the conducted scenario. Specifically, we create a virtual coordinate system in which the coordinates of a gesture trajectory remain unchanged no matter where and how the gesture is conducted. Then, we design a coordinate transformation method to transform the raw doppler spectrogram into the robust intrinsic spectrogram to characterize the intrinsic motion pattern of the gesture. We further feed the intrinsic spectrogram into a deep network to realize gesture recognition. Extensive evaluations on a 77-GHz mmWave testbed show that the proposed method could achieve an average recognize accuracy of 88.4% with ten types of gestures.

Wrist-based hand gesture recognition has the potential to unlock naturalistic human-computer interaction for a vast array of virtual and augmented reality applications. Photoplethysmography (PPG), force myography (FMG), and accelerometry (ACC) have generally been proposed as isolated single sensing modalities for gesture recognition, but any of these alone is inherently limited in the amount of biological information it can collect during finger and hand movements. We thus propose a novel, wrist-based, PPG-FMG-ACC combined sensing approach based on a multi-head attention mechanism fusion convolutional neural network (CNN-AF) for gesture recognition. Nine subjects performed twelve hand gestures involving various wrist and finger postures. Experimental results showed that multi-modal fusion improved classification performance significantly ( p 0.01) compared to any single sensing modality, and the F1-score of the combined PPG-FMG-ACC approach was 40.1% higher than PPG alone, 27.4% higher than ACC alone, and 11.9% higher than FMG alone. To the best of our knowledge, this paper is the first to combine wrist-based PPG, FMG, and ACC signals for hand gesture recognition. These results could serve to inform wrist-based gesture recognition design (e.g., via a smartwatch) and thus expand the capabilities of intuitive and ubiquitous human-machine interaction.

Recent advances in device-free wireless sensing have created the emerging technique of device-free human gesture recognition (DFHGR), which could recognize human gestures by analyzing their shadowing effect on surrounding wireless signals. DFHGR has many potential applications in the fields of human-machine interaction, smart home, intelligent space, etc. State-of-the-art work has achieved satisfactory recognition accuracy when there are a sufficient number of training samples. However, it is time consuming and labor intensive to collect samples, thus how to realize DFHGR under a small training sample set becomes an urgent problem to solve. Motivated by the excellent ability of the generative adversarial network in synthesizing samples, in this article, we explore and exploit the idea of leveraging it to realize virtual samples augmentation. Specifically, we first design a single scenario network with new architecture and better-designed loss function to generate virtual samples using a few number of real samples. Then, we further develop a scenario transferring network to generate virtual samples by utilizing the real samples not only from the current scenario but also from another available scenario as well, which could improve the quality of synthesized samples with the extra knowledge learned from another scenario. We design an mmWave-based DFHGR testbed to test the proposed networks, extensive experimental results demonstrate that the augmented virtual samples are of high quality and facilitate DFHGR systems to achieve better accuracy.

Device-free gesture recognition (DFGR) is a promising sensing technique, which can recognize a gesture by analyzing its influence on surrounding wireless signals. Most of the DFGR systems are designed based on machine learning. However, the recognition performance will drop dramatically when the testing condition is different with the training one. Inspired by the transferrable knowledge learning ability of humans, this paper develops a practical DFGR system based on metalearning to solve the aforementioned problem. Specifically, we design a deep network which could not only learn discriminative deep features, but also learn a transferrable similarity evaluation ability from the training set and apply the learned knowledge to the new testing conditions. Extensive experiments conducted by four users in two scenarios demonstrate that the proposed system could recognize new types of gestures, or gestures performed in new conditions, with an accuracy of more than 90%, using very few number of new samples.

Due to the rapid development of learning-based methods, hyperspectral image classification has achieved remarkable progress. However, since the semantic discrepancy between different land-cover types, the feature distributions of different classes are so nonuniform that the classifier can not measure them with a single rule. In order to consider semantic knowlage in classification, this paper propose a knowledge guided classification method based on hierarchical class tree and deep learning. The proposed method fuses similar classes into several super classes by the knowledge of the confusion matrix, and classify multi-level super classes with different deep networks. Extensive experiments on two hyperspectral images demonstrate that the proposed method can utilize semantic information of different land-cover types, and give better performance than traditional methods.

Device-free activity recognition (DFAR) is a promising technique that detects the activity of a target by analyzing the influence of its existence on surrounding wireless links. It realizes target sensing without the participation or even awareness of the target. The key question of DFAR is how to characterize the influence of the target on wireless links. Existing works mostly utilize statistical features, such as mean and variance in time-domain, and energy as well as entropy in frequency-domain, to characterize the influenced signals. However, statistical features provide only partial information. This paper explores the method on how to characterize the distribution of the signal as a whole. Specifically, we present a novel coherence histogram, which leverages the spatial structural characteristics to better characterize the distribution of the wireless signal. The coherence histogram captures not only the occurrence probability of received signal strength (RSS) measurements, but also the spatial relationship between adjacent RSS measurements as well. Experimental results show that our coherence histogram-based DFAR system could achieve an accuracy of more than 96%, which significantly outperforms other state-of-the-art DFAR systems remarkably.

Due to its promising application in the field of human- machine interaction, letter recognition in the air has drawn considerable attention in recent years. Compared with traditional sensor-based and camera-based methods, letter recognition in the air using channel state information (CSI) is more user-friendly and easy-to-deploy. Unfortunately, due to the limited range of the moving hand and the similarity of different letters, it is difficult to extract discriminative writing patterns for different letters from the noisy environment. In this paper, we design LetFi, a high accuracy letter recognition in the air system, which could detect and recognize the letter written by a user by analyzing its influence on surrounding WiFi signals. Specifically, we design a differential method to extract robust CSI measurements, develop a variance based scheme to detect the start and the end points of the letter writing activity, and propose a coherence histogram based multi-domain feature extraction strategy to extract discriminative features from not only the time domain and frequency domain, but also the spatial structural domain. Extensive experimental results show that the proposed LetFi system could achieve a recognition accuracy of 95% when recognizing the 26 capital letters.

Recent advances in device-free wireless sensing (DFS) have shown that it may eventually evolve traditional wireless networks into smart networks which could sense surrounding target location and activity information without equipping the target with any devices. Despite its promising application prospects, one challenging problem to be solved is that the performance of the DFS system degrades significantly in complex scenarios, such as through-wall and non-line-of-sight (NLOS) scenarios. To alleviate this problem, this paper seeks to explore and exploit more informative features from not only the time domain and frequency domain, but also the spatial structural domain. We partition the time domain and frequency domain measurement matrices into basic structure blocks, adopt self-organizing map networks to cluster the blocks into a number of categories, so as to make it feasible to characterize the block distributions. We further adopt coherence histograms to characterize the distribution of the blocks by considering the spatial relationship between adjacent blocks. Thanks to the additional information provided by the spatial structural domain, extensive experimental results achieved in through-wall and NLOS scenarios confirm the outstanding performance of the proposed multi-domain features based DFS system.

Context awareness plays an important role in many emerging applications, such as mobile computing and smart space. Since FM signal is ubiquitous, it has been recognized as an attractive and promising technique to realize context awareness. When a target is at different locations or performs different activities, it will exert different influence on the FM signal around it. Therefore, it is possible to deduce its location and activity by analysing its influence on the FM signal. However, FM signal is extremely weak and noisy, which makes it a challenging task to achieve high-performance context awareness. In this paper, we propose a new method for improving the performance of an FM-based context-aware system using multidomain features. Specifically, we extract signal features not only from the time domain, but also from the wavelet domain, the frequency domain, and the space domain, and construct robust and discriminative multi-domain features to characterize the FM signal. Furthermore, we also model context awareness as a classification problem and develop a robust iterative sparse representation classification algorithm to efficiently solve this problem. Extensive experiments performed in a 7.2m×10.8m clutter indoor laboratory with one multi-channel FM receiver demonstrate that the proposed schemes could achieve more than 90% accuracy of location estimation and activity recognition when 3 antennas are used.

Device-free wireless localization (DFL) is a promising technique. It can localize and track a target without carrying any electronic device. Compared with traditional received signal strength (RSS)-based DFL, the recently proposed time-of-flight (TOF)-based DFL technique could achieve better performance. However, TOF-based DFL requires that each pair of nodes should perform a pairwise TOF measurement sequentially, which makes it impractical for being utilized in time and energy sensitive applications. Inspired by the fact that a target shadows only a small subset of wireless links, which travel through its spatial impact area, we incorporate link state estimation function into particle filter (PF) framework to predict shadowed links and design a parallel scanning scheme to scan multiple shadowed links simultaneously. The aforementioned methods guarantee that the effective TOF measurements can be acquired timely and energy efficiently. Meanwhile, to achieve reasonable localization performance with these reduced-under-sampled link measurements, we utilize compressive sensing (CS) algorithm to reconstruct the shadowing effect map (SEM). With the map as observation likelihood function, PF algorithm could estimate targets location accurately. Experimental results with an 802.15.4a chirp spread spectrum ranging hardware testbed show that the average running time and energy consumption reduce remarkably.

For realizing robust target tracking with wireless sensor networks in the circumstance where the propagation parameters of the characteristic signal emitted by the target are unknown, a novel tracking algorithm under the particle filter framework is proposed. We propose a scheme to realize particle weight calculation without the prior knowledge about the propagation parameters of the target's characteristic signal. With the use of the monotonic relationship of the distance and the received signal strength, we define the signal characteristic sequence and particle distance sequence and utilize the modified sequence distance between the signal characteristic sequence and the particle distance sequence as the criterion to calculate the particle weight blindly with simple lightweight operations. Simulation results demonstrate the effectiveness of the proposed algorithm. Copyright © 2011 John Wiley & Sons, Ltd.

Due to its ability of realizing device free localization with wireless networks, the radio tomography becomes a promising technique that draws considerable attention. Traditional radio tomography makes use of the received signal strength (RSS) of wireless links to realize location estimation. However, the RSS measurement is particularly sensitive to noise. Inspired by the fact that similar to the RSS, the time-of-flight (TOF) measurement also changes significantly when some objects shadow the wireless link, and the fact that compared with the RSS, the TOF measurement is robust to noise, a novel TOF-based radio tomography is proposed in this paper. With the TOF measurements of the shadowed links as observation information, a modified particle filter algorithm which utilizes the compressive sensing technique to produce the importance distribution of the particle set is proposed, so as to realize localization and tracking with under-sampled measurements by making full use of the space-domain sparse and time-domain gradually changed feature of the location information. The experiments with the 802.15.4a chirp spread spectrum ranging hardware are presented to confirm the proposed scheme.

A compressive sensing-based approach to solve the problem of tracking targets in the deployment area of the wireless networks without the need of equipping the target with a wireless device has been proposed. We present a dynamic statistical model for relating the change of the received signal strength between the node pairs to the spatial location of the target. On the basis of the model, the problem is formulated as a sparse signal reconstruction problem, and we propose a novel Bayesian greedy matching pursuit (BGMP) algorithm to tackle the signal reconstruction problem even from a small set of measurements. The BGMP iteratively seeks the contribution of each pixel for multi-times to compensate for the inaccuracy of the measurement matrix, and builds the enumeration region based on the past estimations to speed up the algorithm and improve its reconstruction performance simultaneously. Experimental results demonstrate the effectiveness of our approach and confirm that the BGMP algorithm could achieve satisfactory localisation and tracking results.

Abstract Benefitting from its ability to estimate the target state's posterior probability density function (PDF) in complex nonlinear and non‐Gaussian circumstance, particle filter (PF) is widely used to solve the target tracking problem in wireless sensor networks. However, the traditional PF algorithm based on sequential importance sampling with re‐sampling will degenerate if the latest observation appear in the tail of the prior PDF or if the observation likelihood is too peaked in comparison with the prior. In this paper, we propose an improved particle filter which makes full use of the latest observation in constructing the proposal distribution. Thequality prediction functionis proposed to measure the quality of the particles, and only the high quality particles are selected and used to generate the coarse proposal distribution. Then, acentroid shift vectoris calculated based on the coarse proposal distribution, which leads the particles move towards the optimal proposal distribution. Simulation results demonstrate the robustness of the proposed algorithm under the challenging background conditions. Copyright © 2010 John Wiley & Sons, Ltd.