A long-standing challenge for pushing sensor-based human activity recognition (HAR) to industrial usage is the distribution shift between training data and testing data: significant variations in data distribution lead to a notable decline in performance. Recently, Large Language Models (LLMs) have demonstrated exceptional generalization capability, which provides a new opportunity to mitigate the distribution shift problem of HAR. However, since LLMs are inherently designed and trained on textual data, their potential to enhance generalization in HAR applications remains an open question. In this paper, we introduce LLM4HAR, a novel LLM-based model to improve cross-domain HAR. LLM4HAR consists of three main modules: (i) the Sensor Data Adaptation module, which aligns IMU signals with LLMs via sensor embedding(ii) the Sensor Knowledge Learning module, which injects sensor knowledge into LLMs for activity recognition, and (iii) the Efficiency Enhancement module, which employs a partial training strategy and reduces the model size by more than 10 times. Extensive evaluations show that LLM4HAR outperforms the existing methods by 13.82% in average F1 score, demonstrating the feasibility and effectiveness of transferring knowledge from pretrained LLMs to enhance HAR. Further, LLM4HAR has been adopted by JD Logistics to support downstream applications such as Courier Welfare Improvement and Map Data Generation.

Human behavior sensing has been receiving growing attention from both academia and industry in recent years. We report - to the best of our knowledge - the first AI-driven nationwide human behavior sensing system called SMILE in urban last-mile delivery. SMILE detects real-time human behaviors with self-supervised sensor data pretraining and uploads detection results to cloud servers with mobile networks. During its full deployment phase at JD Logistics, SMILE is deployed on over 500,000 mobile devices carried by over 300,000 delivery couriers who travel more than 10 million KM every day in 366 Chinese cities. SMILE serves the delivery of 7 billion E-commerce orders every year for more than 500 million customers. SMILE detects walking, upstairs, downstairs, still, and driving behaviors. SMILE has been fully deployed at JD Logistics to support two real-world applications that benefit both the delivery couriers and the logistics platform: (1) workload measurement to improve couriers' welfare; (2) large-scale delivery map data generation to improve the logistics delivery efficiency.

Wireless sensing technologies have shown significant promise in various applications, but their spatial coverage is confined to the vicinity of the transmitters, limiting their applicability in broader environments. In this paper, we introduce an innovative wireless sensing approach based on the globally covered Global Navigation Satellite System (GNSS) signals. While GNSS signals have been widely used in remote sensing to monitor slow changes in the Earth’s surface, like sea level and snow depth, their ability to accurately detect highly dynamic target motions, such as human respiration, gestures, and intrusions, remains unclear. The main challenge arises from the interference brought by the large-scale satellite movement and severe GNSS signal errors. In this study, we present a novel GNSS signal enhancement system named FineSat to address these interference. Specifically, we first utilize polynomial representations to cancel satellite movement interference. Then, based on the analysis of GNSS signal errors, we propose a signal differential processing module to mitigate the errors. We implement our system on commercial devices and validate its performance in three sensing applications: respiration monitoring, gesture recognition, and intrusion detection. Results show that we achieve 0.42 bpm mean absolute error in respiration monitoring, 96.5% average accuracy in gesture recognition, and 98.6% accuracy in intrusion detection.

The Global Navigation Satellite System (GNSS) provides continuous high-precision positioning, enabling many applications such as vehicle navigation and pedestrian monitoring. However, in challenging environments such as urban canyons, positioning accuracy is significantly degraded due to multi-path and non-line-of-sight (NLOS) issues. To tackle this issue, we introduce a single point positioning (SPP) framework based on pseudorange correction, including three modules: feature extraction, pseudorange correction, and positioning model. Heavy pseudorange error is the primary cause of inaccurate localization, and in particular, we introduce GNSSFormer, a transformer-based model to obtain the pseudorange correction values. After preprocessing raw observations of GNSS, we designed a pseudorange correction model GNSSFormer containing a temporal transformer block and a multi-satellite joint spatial transformer block. GNSSFormer extracts multiple features related to both satellites and receivers, learning the complex global relationships between these features and the pseudorange errors to derive correction values. Subsequently, an extended Kalman filter (EKF)-based rauch tung striebel (RTS) smoothing SPP algorithm is employed to determine the location. Validation demonstrates that GNSSFormer significantly enhances SPP performance compared to state-of-the-art algorithms, improving positioning accuracy at least by 29.42% and 28.40% on two open source datasets and 18.91% on real-world dataset.

Cardiovascular diseases (CVDs) are a leading cause of mortality worldwide, highlighting the critical need for accurate and continuous heart health monitoring. Electrocardiograms (ECG), considered as the golden standard for diagnosing and monitoring heart-related conditions, offer precise measurements but require direct skin contact, limiting their practicality for long-term and everyday use. On the other hand, existing RF sensing techniques that analyze signals reflected off the skin struggle to distinguish micro cardiac motions of the heart due to weak motion amplitude and respiration interference at the chest wall. To overcome these limitations, we introduce WiCG, a novel contact-less cardiac motion monitoring system that employs 2.4 GHz Wi-Fi signals to penetrate the chest and detect subtle cardiac movements. A mix-medium Wi-Fi Fresnel zone model is developed to explain the enhanced phase sensitivity of in-body Wi-Fi signals, which is crucial for accurately detecting cardiac motions. By strategically positioning antennas near the heart, WiCG captures ventricular motions effectively. A novel cardiac Doppler method is proposed to suppress phase noise and interference from static paths and extract the time interval between the systole and diastole of the ventricular. Extensive experiments demonstrate that the proposed system can robustly estimate the R-R and Q-T intervals of human cardiac cycles across 21 subjects and different environments with an average accuracy of 99.22% and 92.8%, achieving performance comparable to ECG.

Recent efforts have been made to integrate self-supervised learning (SSL) with the framework of federated learning (FL). One unique challenge of federated self-supervised learning (FedSSL) is that the global objective of FedSSL usually does not equal the weighted sum of local SSL objectives. Consequently, conventional approaches, such as federated averaging (FedAvg), fail to precisely minimize the FedSSL global objective, often resulting in suboptimal performance, especially when data is non-i.i.d.. To fill this gap, we propose a provable FedSSL algorithm, named FedSC, based on the spectral contrastive objective. In FedSC, clients share correlation matrices of data representations in addition to model weights periodically, which enables inter-client contrast of data samples in addition to intra-client contrast and contraction, resulting in improved quality of data representations. Differential privacy (DP) protection is deployed to control the additional privacy leakage on local datasets when correlation matrices are shared. We provide theoretical analysis on convergence and extra privacy leakage, and conduct numerical experiments to justify the effectiveness of our proposed algorithm.

Error correcting output code (ECOC) is a classic method that encodes binary classifiers to tackle the multi-class classification problem in decision trees and neural networks. Among ECOCs, the one-hot code has become the default choice in modern deep neural networks (DNNs) due to its simplicity in decision making. However, it suffers from a significant limitation in its ability to achieve high robust accuracy, particularly in the presence of weight errors. While recent studies have experimentally demonstrated that the non-one-hot ECOCs with multi-bits error correction ability, could be a better solution, there is a notable absence of theoretical foundations that can elucidate the relationship between codeword design, weight-error magnitude, and network characteristics, so as to provide robustness guarantees. This work is positioned to bridge this gap through the lens of neural tangent kernel (NTK). We have two important theoretical findings: 1) In clean models (without weight errors), utilizing one-hot code and non-one-hot ECOC is akin to altering decoding metrics from $l_2$ distance to Mahalanobis distance. 2) In non-clean models (with weight errors), if the normalized distance exceeds a threshold, then non-clean DNNs can reach the clean model's accuracy as long as the code length approaches infinity. This threshold is determined by DNN architecture (e.g. layer number, activation), weight error magnitude, and the distance between the output and the nearest codeword. Based on these findings, we further demonstrate how to practically use them to identify optimal ECOCs for simple tasks (short-code ECOCs) and complex tasks (long-code ECOCs), by balancing the code orthogonality (as per finding 1) and code distance (as per finding 2). Extensive experimental results across four datasets and four DNN models validate the superior performance of constructed codes, guided by our findings, compared to existing ECOCs. To our best knowledge, this is the first work that provides theoretical explanations for the effectiveness of ECOCS and offers associated design guidance for optimal ECOCs specifically tailored to DNNs.

While a large range of sensing applications such as activity sensing and vital sign monitoring have been realized with WiFi sensing, using commercial WiFi devices to obtain fine-grained size information of objects remains challenging due to the narrow bandwidth of WiFi. Very recent studies attempted to measure object sizes using WiFi signals. However, these systems are still far from practical with a lot of limitations including requiring multiple transceiver pairs and can only measure one-dimensional size, hindering their real-life adoption. Also, these systems rely on Channel State Information (CSI) to work, which is only available on few commercial WiFi cards. In this work, we propose to employ a new channel data, i.e., Beamforming Feedback Information (BFI), widely available on almost all new generation WiFi cards for fine-grained size measurement. Through thoroughly analyzing the mathematical relationship between BFI and CSI, we show how to use BFI to achieve fine-grained size measurement. We propose a novel method to accurately measure the two-dimensional size of an object using a single transceiver pair by identifying the positions of singularities when the object passes through the diffraction zone of the transceiver pair. Experiment results show that Wi2DMeasure can accurately measure the two-dimensional size of objects under various conditions, achieving a small median error of only 3.7 mm.

Although WiFi-based contactless sensing has made significant progress in the past decade, most prior work still focus on the reflection zone far from WiFi transceivers, while few studies explore the diffraction zone near transceivers. Additionally, previous diffraction models only consider the CSI amplitude signal and ignore the impact of multipath. In this work, we develop an accurate diffraction model to characterize the relationship between both CSI amplitude and phase and target's movement in the diffraction zone. We further put forward the deformation forms of the model under static multipath conditions and find that the CSI patterns vary significantly with multipath. Consequently, the common assumption of a one-to-one mapping between CSI patterns and activities in existing work fails due to multipaths, degrading sensing performance when multipath changes. To address this challenge, we propose to extract a relative change pattern from CSI signals to recover the one-to-one mapping relations and eliminate the impact of static multipath. Extensive experiments under various multipath conditions demonstrate an accuracy higher than 96% for the coarse-grained intrusion detection and an average error rate of 0.6 bpm for the fine-grained respiration monitoring.

Error correcting output codes (ECOCs) have been proposed to improve the robustness of deep neural networks (DNNs) against hardware defects of DNN hardware accelerators. Unfortunately, existing efforts suffer from drawbacks that would greatly impact their practicality: 1) robust accuracy (with defects) improvement at the cost of degraded clean accuracy (without defects); 2) no guarantee on better robust or clean accuracy using stronger ECOCs. In this paper, we first shed light on the connection between these drawbacks and error correlation, and then propose a novel comprehensive error decorrelation framework, namely COLA. Specifically, we propose to reduce inner layer feature error correlation by 1) adopting a separated architecture, where the last portions of the paths to all output nodes are separated, and 2) orthogonalizing weights in common DNN layers so that the intermediate features are orthogonal with each other. We also propose a regularization technique based on total correlation to mitigate overall error correlation at the outputs. The effectiveness of COLA is first analyzed theoretically, and then evaluated experimentally, e.g. up to 6.7% clean accuracy improvement compared with the original DNNs and up to 40% robust accuracy improvement compared to the state-of-the-art ECOC-enhanced DNNs.

Implementing deep neural networks (DNNs) in hardware is challenging due to the requirements of huge memory and computation associated with DNNs' primary operation—matrix-vector multiplications (MVMs). Memristive crossbar shows great potential to accelerate MVMs by leveraging its capability of in-memory computation. However, one critical obstacle to such a technique is potentially significant inference accuracy degradation caused by two primary sources of errors—the variations during computation and stuck-at-faults (SAFs). To overcome this obstacle, we propose a set of dedicated schemes to significantly enhance its tolerance against these errors. First, a minimum mean square error (MMSE) based denoising scheme is proposed to diminish the impact of variations during computation in the intermediate layers. To the best of our knowledge, this is the first work considering denoising in the intermediate layers without extra crossbar resources. Furthermore, MMSE early denoising not only stabilizes the crossbar computation results but also mitigates errors caused by low resolution analog-to-digital converters. Second, we propose a weights-to-crossbar mapping scheme by inverting bits to mitigate the impact of SAFs. The effectiveness of the proposed bit inversion scheme is analyzed theoretically and demonstrated experimentally. Finally, we propose to use L1 regularization to increase the network sparsity, as a greater sparsity not only further enhances the effectiveness of the proposed bit inversion scheme, but also facilitates other early denoising mechanisms. Experimental results show that our schemes can achieve 40%-78% accuracy improvement, for the MNIST and CIFAR10 classification tasks under different networks.

With low-density parity-check (LDPC) code and polar code selected as the standard codes for 5G eMBB scenario, one challenge is how to improve the hardware efficiency when both decoders are required by one system. Since LDPC and polar codes can be decoded with belief propagation (BP) algorithms, this similarity allows us to design a reconfigurable decoder, which can decode both codes at the cost of only one decoder. Numerical and implementation results are also given in this paper to show that the proposed decoder achieves higher hardware efficiency than stand-alone LDPC or polar decoder, without harming the error performance.