The rapid expansion of the Internet of Things (IoT) has introduced new challenges in Human Activity Recognition (HAR), particularly in dynamic environments where new and unforeseen activities emerge. Traditional HAR models, relying on predefined labels, struggle to adapt to these scenarios, highlighting the need for zero-shot learning (ZSL) approaches that can generalize beyond fixed training categories. Recent advances in large language models (LLMs) have demonstrated remarkable zero-shot capability in textual and visual domains. However, extending this ability to IoT sensors is substantially more challenging due to their heterogeneous modalities, diverse data structures, and limited semantic annotations. In this paper, we propose TENT (IoT-sEnsorslanguage alignmEnt pre-Training), a novel framework that constructs a unified sensor-language semantic space for zero-shot HAR. Instead of aligning each sensor individually to text, TENT jointly aligns multiple heterogeneous modalities with language, treating them as peers rather than anchors. This balanced multi-modal alignment allows sensors to mutually regularize one another while being grounded in linguistic semantics, transforming heterogeneity from a barrier into a strength. To further enrich the semantic space, TENT incorporates detailed activity descriptions and learnable prompts, enhancing adaptability to unseen activities. Extensive experiments across datasets and evaluation protocols demonstrate that TENT not only achieves robust recognition of both seen and unseen activities but also significantly outperforms existing vision-language and sensor-language baselines, surpassing them by over 20% on zero-shot HAR tasks. These results establish TENT as a new paradigm for generalizable IoT representation learning.

Cloud-based Large Language Model (LLM) services often face challenges in achieving low inference latency and meeting Service Level Objectives (SLOs) under dynamic request patterns. Speculative decoding, which exploits lightweight models for drafting and LLMs for verification, has emerged as a compelling technique to accelerate LLM inference. However, existing speculative decoding solutions often fail to adapt to fluctuating workloads and dynamic system environments, resulting in impaired performance and SLO violations. In this paper, we introduce AdaSpec, an efficient LLM inference system that dynamically adjusts speculative strategies according to real-time request loads and system configurations. AdaSpec proposes a theoretical model to analyze and predict the efficiency of speculative strategies across diverse scenarios. Additionally, it implements intelligent drafting and verification algorithms to maximize performance while ensuring high SLO attainment. Experimental results on real-world LLM service traces demonstrate that AdaSpec consistently meets SLOs and achieves substantial performance improvements, delivering up to 66% speedup compared to state-of-the-art speculative inference systems. The source code is publicly available at https://github.com/cerebellumking/AdaSpec

In this work, we are interested in achieving both high text controllability and whole-body appearance consistency in the generation of personalized human characters. We propose a novel framework, named SerialGen, which is a serial generation method consisting of two stages: first, a standardization stage that standardizes reference images, and then a personalized generation stage based on the standardized reference. Furthermore, we introduce two modules aimed at enhancing the standardization process. Our experimental results validate the proposed framework’s ability to produce personalized images that faithfully recover the reference image’s whole-body appearance while accurately responding to a wide range of text prompts. Through thorough analysis, we highlight the critical contribution of the proposed serial generation method and standardization model, evidencing enhancements in appearance consistency between reference and output images and across serial outputs generated from diverse text prompts. The term "Serial" in this work carries a double meaning: it refers to the two-stage method and also underlines our ability to generate serial images with consistent appearance throughout.

Smartphones with multi-camera systems, featuring cameras with varying field-of-views (FoVs), are increasingly common. This variation in FoVs results in content differences across videos, paving the way for an innovative approach to video super-resolution (VSR). This method enhances the VSR performance of lower resolution (LR) videos by leveraging higher resolution reference (Ref) videos. Previous works [14, 15 ], which operate on this principle, generally expand on traditional VSR models by combining LR and Ref inputs over time into a unified stream. However, we can expect that better results are obtained by independently aggre-gating these Ref image sequences temporally. Therefore, we introduce an improved method, RefVSR++, which performs the parallel aggregation of LR and Ref images in the temporal direction, aiming to optimize the use of the available data. RefVSR++ also incorporates improved mechanisms for aligning image features over time, crucial for effective VSR. Our experiments demonstrate that RefVSR++ outper-forms previous works by over 1dB in PSNR, setting a new benchmark in the field.

Over the past few years, cross-domain recommendation has gained great attention to resolve the cold-start issue. Many existing cross-domain recommendation methods model a preference bridge between the source and target domains to transfer preferences by the overlapping users. However, when there are insufficient cross-domain users available to bridge the two domains, it will negatively impact the recommender system's accuracy (ACC) and performance. Therefore, in this article, we propose to create a link between the source and the target domains by leveraging knowledge graph (KG) as the auxiliary information, and propose a novel knowledge-reinforced cross-domain recommendation (KR-CDR) method. First of all, we construct a new cross-domain KG (CDKG) by using the KGs that represent the source and target domains, respectively. Additionally, we employ reinforcement learning (RL) with meta learning on CDKG to discover meta-paths between the source and target domains. With these meta-paths, we obtain meta-path aggregated embedding vectors for cold-start users. Ultimately, the predicted rating can be acquired from the user meta-path aggregated embedding vector and item embedding vector. Experiments carried out on five real-world datasets show that the proposed method performs better than the state-of-the-art methods.

Abstract With the development of automatic driving and path planning technology, predicting the moving trajectory of pedestrians in dynamic scenes has become one of key and urgent technical problems. However, most of the existing techniques regard all pedestrians in the scene as equally important influence on the predicted pedestrian's trajectory, and the existing methods which use sequence‐based time‐series generative models to obtain the predicted trajectories, do not allow for parallel computation, it will introduce a significant computational overhead. A new social trajectory prediction network, Social‐ATPGNN which integrates both temporal information and spatial one based on ATPGNN is proposed. In space domain, the pedestrians in the predicted scene are formed into an undirected and non fully connected graph, which solves the problem of homogenisation of pedestrian relationships, then, the spatial interaction between pedestrians is encoded to improve the accuracy of modelling pedestrian social consciousness. After acquiring high‐level spatial data, the method uses Temporal Convolutional Network which could perform parallel calculations to capture the correlation of time series of pedestrian trajectories. Through a large number of experiments, the proposed model shows the superiority over the latest models on various pedestrian trajectory datasets.

Deep neural networks have empowered accurate device-free human activity recognition, which has wide applications. Deep models can extract robust features from various sensors and generalize well even in challenging situations such as data-insufficient cases. However, these systems could be vulnerable to input perturbations, i.e., adversarial attacks. We empirically demonstrate that both black-box Gaussian attacks and modern adversarial white-box attacks can render their accuracies to plummet. In this paper, we first point out that such phenomenon can bring severe safety hazards to device-free sensing systems, and then propose a novel learning framework, SecureSense, to defend common attacks. SecureSense aims to achieve consistent predictions regardless of whether there exists an attack on its input or not, alleviating the negative effect of distribution perturbation caused by adversarial attacks. Extensive experiments demonstrate that our proposed method can significantly enhance the model robustness of existing deep models, overcoming possible attacks. The results validate that our method works well on wireless human activity recognition and person identification systems. To the best of our knowledge, this is the first work to investigate adversarial attacks and further develop a novel defense framework for wireless human activity recognition in mobile computing research.

4D human perception plays an essential role in a myriad of applications, such as home automation and metaverse avatar simulation. However, existing solutions which mainly rely on cameras and wearable devices are either privacy intrusive or inconvenient to use. To address these issues, wireless sensing has emerged as a promising alternative, leveraging LiDAR, mmWave radar, and WiFi signals for device-free human sensing. In this paper, we propose MM-Fi, the first multi-modal non-intrusive 4D human dataset with 27 daily or rehabilitation action categories, to bridge the gap between wireless sensing and high-level human perception tasks. MM-Fi consists of over 320k synchronized frames of five modalities from 40 human subjects. Various annotations are provided to support potential sensing tasks, e.g., human pose estimation and action recognition. Extensive experiments have been conducted to compare the sensing capacity of each or several modalities in terms of multiple tasks. We envision that MM-Fi can contribute to wireless sensing research with respect to action recognition, human pose estimation, multi-modal learning, cross-modal supervision, and interdisciplinary healthcare research.

As an important biomarker for human identification, human gait can be collected at a distance by passive sensors without subject cooperation, which plays an essential role in crime prevention, security detection, and other human identification applications. Presently, most research works are based on cameras and computer vision techniques to perform gait recognition. However, vision-based methods are not reliable when confronting poor illuminations, leading to degrading performances. In this article, we propose a novel multimodal gait recognition method, namely, GaitFi, which leverages WiFi signals and videos for human identification. In GaitFi, channel state information (CSI) that reflects the multipath propagation of WiFi is collected to capture human gaits, while videos are captured by cameras. To learn robust gait information, we propose a lightweight residual convolution network (LRCN) as the backbone network and further propose the two-stream GaitFi by integrating WiFi and vision features for the gait retrieval task. The GaitFi is trained by the triplet loss and classification loss on different levels of features. Extensive experiments are conducted in the real world, which demonstrates that the GaitFi outperforms state-of-the-art gait recognition methods based on single WiFi or camera, achieving 94.2% for human identification tasks of 12 subjects.

Wi-Fi sensing technology has shown superiority in smart homes among various sensors for its cost-effective and privacy-preserving merits. It is empowered by channel state information (CSI) extracted from Wi-Fi signals and advanced machine learning models to analyze motion patterns in CSI. Many learning-based models have been proposed for kinds of applications, but they severely suffer from environmental dependency. Though domain adaptation methods have been proposed to tackle this issue, it is not practical to collect high-quality, well-segmented, and balanced CSI samples in a new environment for adaptation algorithms, but randomly captured CSI samples can be easily collected. In this article, we first explore how to learn a robust model from these low-quality CSI samples, and propose AutoFi, an annotation-efficient Wi-Fi sensing model based on a novel geometric self-supervised learning algorithm. The AutoFi fully utilizes unlabeled low-quality CSI samples that are captured randomly, and then transfers the knowledge to specific tasks defined by users, which is the first work to achieve cross-task transfer in Wi-Fi sensing. The AutoFi is implemented on a pair of Atheros Wi-Fi APs for evaluation. The AutoFi transfers knowledge from randomly collected CSI samples into human gait recognition and achieves state-of-the-art performance. Furthermore, we simulate cross-task transfer using public data sets to further demonstrate its capacity for cross-task learning. For the UT-HAR and Widar data sets, the AutoFi achieves satisfactory results on activity recognition and gesture recognition without any prior training. We believe that AutoFi takes a huge step toward automatic Wi-Fi sensing without any developer engagement. Our codes have been included inhttps://github.com/xyanchen/Wi-Fi-CSI-Sensing-Benchmark.

In the metaverse, digital avatar plays an important role in representing human beings for various interaction with virtual objects and environments, which puts a high demand on effective pose estimation. Though camera-based solutions yield remarkable performance, they encounter privacy issues and degraded performance caused by varying illumination, especially in the smart home. In this article, we propose a WiFi-based Internet of Things-enabled human pose estimation scheme for metaverse avatar simulation, namely, MetaFi++. Specifically, WPFormer is designed with a shared convolutional module and a Transformer block to map the channel state information of WiFi signals to human pose landmarks, effectively exploring spatial information of human pose through self-attention. It is enforced to learn the annotations from the accurate computer vision model, thus achieving cross-modal supervision. Due to the ubiquitous existence of WiFi and robustness to various illumination conditions, WiFi-based human poses are suitable to instruct the movement of digital avatars in the metaverse, promoting avatar applications in smart homes. The experiments are conducted in the real world, and the results show that the MetaFi++ achieves very high performance with a PCK@50 of 97.30%. Our codes are available inhttps://github.com/pridy999/metafi_pose_estimation.

Unsupervised domain adaptation methods have been proposed to tackle the problem of covariate shift by minimizing the distribution discrepancy between the feature embeddings of source domain and target domain. However, the standard evaluation protocols assume that the conditional label distributions of the two domains are invariant, which is usually not consistent with the real-world scenarios such as long-tailed distribution of visual categories. In this article, the imbalanced domain adaptation (IDA) is formulated for a more realistic scenario where both label shift and covariate shift occur between the two domains. Theoretically, when label shift exists, aligning the marginal distributions may result in negative transfer. Therefore, a novel cluster-level discrepancy minimization (CDM) is developed. CDM proposes cross-domain similarity learning to learn tight and discriminative clusters, which are utilized for both feature-level and distribution-level discrepancy minimization, palliating the negative effect of label shift during domain transfer. Theoretical justifications further demonstrate that CDM minimizes the target risk in a progressive manner. To corroborate the effectiveness of CDM, we propose two evaluation protocols according to the real-world situation and benchmark existing domain adaptation approaches. Extensive experiments demonstrate that negative transfer does occur due to label shift, while our approach achieves significant improvement on imbalanced datasets, including Office-31, Image-CLEF, and Office-Home.

WiFi technology has been applied to various places due to the increasing requirement of high-speed Internet access. Recently, besides network services, WiFi sensing is appealing in smart homes since it is device free, cost effective and privacy preserving. Though numerous WiFi sensing methods have been developed, most of them only consider single smart home scenario. Without the connection of powerful cloud server and massive users, large-scale WiFi sensing is still difficult. In this article, we first analyze and summarize these obstacles, and propose an efficient large-scale WiFi sensing framework, namely, EfficientFi. The EfficientFi works with edge computing at WiFi access points and cloud computing at center servers. It consists of a novel deep neural network that can compress fine-grained WiFi channel state information (CSI) at edge, restore CSI at cloud, and perform sensing tasks simultaneously. A quantized autoencoder and a joint classifier are designed to achieve these goals in an end-to-end fashion. To the best of our knowledge, the EfficientFi is the first Internet of Things-cloud-enabled WiFi sensing framework that significantly reduces communication overhead while realizing sensing tasks accurately. We utilized human activity recognition (HAR) and identification via WiFi sensing as two case studies, and conduct extensive experiments to evaluate the EfficientFi. The results show that it compresses CSI data from 1.368 Mb/s to 0.768 kb/s with extremely low error of data reconstruction and achieves over 98% accuracy for HAR.

Deep models have achieved prominent results in pattern recognition tasks, especially computer vision and natural language processing. However, the dataset bias caused by the distribution discrepancy between the training and testing data hinders the generalization ability of deep models. Though many domain adaptation approaches have been proposed to mitigate such negative effect, most of them improve the transferability of features by aligning global distributions of deep models. Few researchers pay attention to the versatility of deep features which can play a vital role in cross-domain recognition. In this paper, we propose to enrich the classic deep learning models by capturing high-low-frequency information and multi-scale features, which deal with the domain shift that cannot be easily addressed by merely feature-level alignment. The Hierarchical Transfer Network (HTN) leverages octave convolution, pyramid features, and self-attention mechanism for revamping the classic models, which can be further integrated with any domain alignment approaches by replacing the feature extractor with the proposed HTN. Extensive experiments have been conducted on three public domain adaptation benchmarks. The results show that the proposed HTN can effectively improve adversarial-based, statistics-based, and norm-based domain adaptation approaches, achieving competitive performance without involving model complexity.

Deep neural networks (DNNs) have made significant advances in computer vision and sensor-based smart sensing. DNNs achieve prominent results based on standard data sets and powerful servers, whereas, in real applications with domain-shift data and resource-constrained environments such as Internet-of-Things (IoT) devices in the edge computing, DNNs are likely to have degraded performance in terms of accuracy and efficiency. To this end, we develop the MobileDA framework that learns transferable features while keeping the simple structure of the deep model. Our method allows a novel teacher network trained in the server to distill the knowledge for a student network running in the edge device, which is achieved by a cross-domain distillation. Leveraging unlabeled data in the new environment, our student model amends the feature learning to be domain invariant, then being our objective model running in the edge device. Our approach is evaluated on a challenging IoT-based WiFi gesture recognition scenario, and three classic visual adaptation benchmarks. The empirical studies corroborate the effectiveness of distillation for domain transfer, and the overall results show that our model achieves state-of-the-art performance merely using a simple network.

Location-based service (LBS) has become an indispensable part of our daily lives. Realizing accurate LBS in indoor environments is still a challenging task. WiFi fingerprinting-based indoor positioning system (IPS) has achieved encouraging results recently, but the time and labor overhead of constructing a dense WiFi radio map remains the key bottleneck that hinders it for real-world large-scale implementation. In this article, we propose WiGAN an automatic fine-grained indoor ratio map construction and the adaptation scheme empowered by the Gaussian process regression conditioned least-squares generative adversarial networks (GPR-GANs) with a mobile robot. First, we develop a mobile robotic platform that constructs the spatial map and radio map simultaneously in the easily accessed free space. GPR-GAN first establishes a Gaussian process regression (GPR) model using the real received signal strength (RSS) measurements collected by our robotic platform via LiDAR SLAM in the free space. Then, the outputs of the GPR are adopted as the input of GAN's generator. The learning objective of GAN is to synthesize realistic RSS data in a constrained space where it has not been covered and model the irregular RSS distributions in complex indoor environments. Real-world experiments were conducted in a real-world indoor environment, which confirms the feasibility, high accuracy, and superiority of WiGAN over existing solutions in terms of both RSS estimation accuracy and localization accuracy.

We propose a novel domain adaptation framework, namely Consensus Adversarial Domain Adaptation (CADA), that gives freedom to both target encoder and source encoder to embed data from both domains into a common domaininvariant feature space until they achieve consensus during adversarial learning. In this manner, the domain discrepancy can be further minimized in the embedded space, yielding more generalizable representations. The framework is also extended to establish a new few-shot domain adaptation scheme (F-CADA), that remarkably enhances the ADA performance by efficiently propagating a few labeled data once available in the target domain. Extensive experiments are conducted on the task of digit recognition across multiple benchmark datasets and a real-world problem involving WiFi-enabled device-free gesture recognition under spatial dynamics. The results show the compelling performance of CADA versus the state-of-the-art unsupervised domain adaptation (UDA) and supervised domain adaptation (SDA) methods. Numerical experiments also demonstrate that F-CADA can significantly improve the adaptation performance even with sparsely labeled data in the target domain.

We propose a gesture recognition system that leverages existing WiFi infrastructures and learns gestures from channel state information (CSI) measurements. Having developed an innovative OpenWrt-based platform for commercial WiFi devices to extract CSI data, we propose a novel deep Siamese representation learning architecture for one-shot gesture recognition. Technically, our model extends the capacity of spatio-temporal patterns learning for the standard Siamese structure by incorporating convolutional and bidirectional recurrent neural networks. More importantly, the representation learning is ameliorated by our Siamese framework and transferable pairwise loss which helps to remove structured noise, such as individual heterogeneity and various measurement conditions during domain-different training. Meanwhile, our Siamese model also enables one-shot learning for higher availability in reality. We prototype our system on commercial WiFi routers. The experiments demonstrate that our model outperforms state-of-the-art solutions for temporal-spatial representation learning and achieves satisfactory results under one-shot conditions.

In the research of smart buildings, human activity recognition is an important cornerstone for numerous emerging applications. Although several sensing techniques have been proposed for human activity identification, they require either the user instrumentation or additional infrastructure, that are inconvenient, privacy-intrusive, and expensive. To address these issues, a WiFi-enabled device-free human activity recognition system, namely SmartSense, is proposed in this paper. By upgrading commercial WiFi routers with our designed firmware, fine-grained channel state information (CSI) from PHY layer can be directly extracted from the existing WiFi traffic. In this paper, we propose SmartSense, a device-free human activity recognition system that only leverages existing commercial off-the-shelf WiFi routers. By exploiting the prevalence of WiFi infrastructure in buildings, we developed a novel CSI-enabled Internet of Things platform to collect the CSI measurements from regular data frames. To identify different human activities, a novel machine learning tool, namely multiple kernel semi-representation learning (MKSRL) method is established. MKSRL allows the input of expert domain knowledge in a flexible way and conducts automatic and effective multikernel representation learning for the activity recognition task. Each stage of MKSRL is computationally efficient and theoretically guaranteed, and they can be integrated seamlessly within the reproducing kernel framework for the overall information extraction, representation, and fusion. We conducted experiments to comprehensively evaluate the performance of SmartSense in three common indoor environments. Experimental results validate that SmartSense can provide an activity recognition accuracy of 98%, which achieves significant performance gain over the existing methods.

A fundamental building block toward personalized location-based service and context-aware service in smart buildings is the knowledge about the identity and mobility of users in indoor environments. Conventional user identification systems require the deployment of dedicated infrastructure or the active user involvement. Motivated by the widespread usage of the WiFi-enabled mobile device (MD), e.g., people usually carry at least one MD in their daily lives, in this paper, we propose WinDUA, a WiFi-enabled nonintrusive device and user association scheme to infer user identity and mobility via a novel unsupervised association learning algorithm. First, we utilize our WiFi-based indoor positioning system to obtain the historical location data of each MD using only existing WiFi infrastructure in a nonintrusive manner. Then, we classify all the MDs into two categories: 1) static device (SD) and 2) mobile phone (MP), according to their location variations and overnight presences. Subsequently, we estimate the correct mapping between each SD and its user through hierarchical clustering and location similarity matching between its location and user's personal space. Finally, we make possible pairs of MP and SD according to their duration of coexistence as well as the historical location similarity to associate the owner of each MP. Real-world experiments are conducted in an office, verifying that WinDUA is able to associate the MD to the correct users in a nonintrusive and unsupervised manner.

In this study we consider the problem of outlier detection with multiple co-evolving time series data. To capture both the temporal dependence and the inter-series relatedness, a multi-task non-parametric model is proposed, which can be extended to data with a broader exponential family distribution by adopting the notion of Bregman divergence. Albeit convex, the learning problem can be hard as the time series accumulate. In this regards, an efficient randomized block coordinate descent (RBCD) algorithm is proposed. The model and the algorithm is tested with a real-world application, involving outlier detection and event analysis in power distribution networks with high resolution multi-stream measurements. It is shown that the incorporation of inter-series relatedness enables the detection of system level events which would otherwise be unobservable with traditional methods.

We propose AutoID, a human identification system that leverages the measurements from existing WiFi-enabled Internet of Things (IoT) devices and produces the identity estimation via a novel sparse representation learning technique. The key idea is to use the unique fine-grained gait patterns of each person revealed from the WiFi Channel State Information (CSI) measurements, technically referred to as shapelet signatures, as the "fingerprint" for human identification. For this purpose, a novel OpenWrt-based IoT platform is designed to collect CSI data from commercial IoT devices. More importantly, we propose a new optimization-based shapelet learning framework for tensors, namely Convex Clustered Concurrent Shapelet Learning (C3SL), which formulates the learning problem as a convex optimization. The global solution of C3SL can be obtained efficiently with a generalized gradient-based algorithm, and the three concurrent regularization terms reveal the inter-dependence and the clustering effect of the CSI tensor data. Extensive experiments are conducted in multiple real-world indoor environments, showing that AutoID achieves an average human identification accuracy of 91% from a group of 20 people. As a combination of novel sensing and learning platform, AutoID attains substantial progress towards a more accurate, cost-effective and sustainable human identification system for pervasive implementations.

In the era of Internet of Things (IoT), human activity recognition is becoming the vital underpinning for a myriad of emerging applications in smart home and smart buildings. Existing activity recognition approaches require either the deployment of extra infrastructure or the cooperation of occupants to carry dedicated devices, which are expensive, intrusive and inconvenient for pervasive implementation. In this paper, we propose DeepSense, a device-free human activity recognition scheme that can automatically identify common activities via deep learning using only commodity WiFi-enabled IoT devices. We design a novel OpenWrt-based IoT platform to collect Channel State Information (CSI) measurements from commercial IoT devices. Moreover, an innovative deep learning framework, Autoencoder Long-term Recurrent Convolutional Network (AE-LRCN), is proposed. It consists of an autoencoder module, a convolutional neural network (CNN) module and a long short-term memory (LSTM) module, which aims to sanitize the noise in raw CSI data, extract high-level representative features and reveal the inherent temporal dependencies among data for accurate human activity recognition, respectively. All the hyperparameters in AE-LRCN are fine-tuned end-to-end automatically. Extensive experiments are conducted in typical indoor environments and the experimental results demonstrate that DeepSense outperforms existing methods and achieves a 97.6% activity recognition accuracy without human intervention.

Accurate human gesture recognition is becoming a cornerstone for myriad emerging applications in human-computer interaction. Existing gesture recognition systems either require dedicated extra infrastructure or user's active cooperation. Although some WiFi-enabled gesture recognition systems have been proposed, they are vulnerable to environmental dynamics and rely on the tedious data re-labeling and expert knowledge each time being implemented in a new environment. In this paper, we propose a WiFi- enabled device-free adaptive gesture recognition scheme, WiADG, that is able to identify human gestures accurately and consistently under environmental dynamics via adversarial domain adaptation. Firstly, a novel OpenWrt-based IoT platform is developed, enabling the direct collection of Channel State Information (CSI) measurements from commercial IoT devices. After constructing an accurate source classifier with labeled source CSI data via the proposed convolutional neural network in the source domain (original environment), we design an unsupervised domain adaptation scheme to reduce the domain discrepancy between the source and the target domain (new environment) and thus improve the generalization performance of the source classifier. The domain- adversarial objective is to train a generator (target encoder) to map the unlabeled target data to a domain invariant latent feature space so that a domain discriminator cannot distinguish the domain labels of the data. In the phase of implementation, we utilize the trained target encoder to map the target CSI frame to the latent feature space and use the source classifier to identify various gestures performed by the user. We implement WiADG on commercial WiFi routers and conduct experiments in multiple indoor environments. The results validate that WiADG achieves 98% gesture recognition accuracy in the original environment. Furthermore, the proposed unsupervised adversarial domain adaptation is able to enhance the recognition accuracy of WiADG by 25% on average without the needs of labeled data collection and new classifier generation when implements it in new environments.

Human gesture recognition plays a critical role in numerous applications of human-computer interaction. By analyzing how gesture alters the WiFi propagation among WiFi-enabled IoT devices to identify the gestures in a device-free manner could be a promising solution. However, existing methods require tedious data collection and labeling process each time being implemented in a new environment. The classifier constructed by SVM or random forest is vulnerable to spatial dynamics. In this paper, we proposed JADA, a novel unsupervised Joint adversarial domain adaptation (JADA) scheme that realizes accurate and resilient WiFi-enabled device-free gesture recognition without collecting and labeling training data in new environments. After constructing a source encoder and a source classifier in the source domain by convolutional neural network, JADA trains a target encoder and also fine-tunes the source encoder through adversarial learning to map both unlabeled target data and labeled source data to a domain-invariant feature space such that a domain discriminator cannot distinguish the domain labels of the data. After training a shared classifier with the labeled source data while fixing the parameters of the source encoder, we employ the trained target encoder to embed the test target samples into the domain-invariant feature space and infer its class using the shared classifier. We develop a novel Channel State Information (CSI) enabled IoT platform that could obtain fine-grained CSI time series data directly from IoT devices and transform them into CSI frames. Real-world experiments with COTS WiFi routers were conducted in 2 indoor environments. The experimental results demonstrate that JADA achieves 98.75% gesture recognition accuracy in the original environment. Moreover, when the environmental scenario is altered, it is able to reduce the domain discrepancy across domains without collecting any labeled data in the new context.

Device-free activity recognition is appealing in smart home applications. It not only is convenient, but also causes no privacy concern, as compared to other activity recognition techniques such as the vision based technique. Existing WiFi-based methods have achieved high accuracy in static circumstances but have limitations in adapting changes in environment and activities locations. In this paper, we propose a fine-grained adaptive location-independent activity recognition system (FALAR) which leverages WiFi signals to characterize and recognize common activities regardless of inconsistency of mutative surroundings. FALAR applies fine-grained channel state information (CSI) to achieve accurate recognitions. To address the issue of environmental changes, we present a Kernel Density Estimation (KDE) based motion extraction method and a coarse-to-fine search strategy for speedy processing. After a denoising scheme, we introduce Class Estimated Basis Space Singular Value Decomposition (CSVD) to efface the static path in the background, and use nonnegative matrix factorization to distinguish various activities by looking into the signal profiles. We evaluate FALAR using two commodity WiFi routers in a typical office environment. Our results show that it achieves remarkable performance.

Intelligent occupancy sensing is becoming a vital underpinning for various emerging applications in smart homes, such as security surveillance and human behavior analysis. However, prevailing approaches mainly rely on video camera, ambient sensors, or wearable devices, which either requires arduous deployment or arouses privacy concerns. In this paper, we present a novel real-time, device-free, and privacy-preserving WiFi-enabled Internet of Things platform for occupancy sensing, which can promote a myriad of emerging applications. It is designed to achieve an optimal tradeoff between performance and scalability. Our system empowers commercial off-the-shelf WiFi routers to collect channel state information (CSI) measurements and provides an efficient cloud server for computing via a lightweight communication protocol. To demonstrate the usefulness of our platform, an occupancy detection system is developed by exploiting the CSI curve of human presence. Furthermore, we also design an innovative activity recognition system based on our platform and machine learning techniques with high availability and extensibility. In the evaluation, the experimental results show that our platform enables these applications efficiently, with the accuracy of 96.8% and 90.6% in terms of occupancy detection and recognition, respectively.

Smart buildings today are aimed at providing safe, healthy, comfortable, affordable, and beautiful spaces in a carbon and energy-efficient way. They are emerging as complex cyber-physical systems with humans in the loop. Cost, the need to cope with increasing functional complexity, flexibility, fragmentation of the supply chain, and time-to-market pressure are rendering the traditional heuristic and ad hoc design paradigms inefficient and insufficient for the future. In this paper, we present a platform-based methodology for smart building design. Platform-based design (PBD) promotes the reuse of hardware and software on shared infrastructures, enables rapid prototyping of applications, and involves extensive exploration of the design space to optimize design performance. In this paper, we identify, abstract, and formalize components of smart buildings, and present a design flow that maps high-level specifications of desired building applications to their physical implementations under the PBD framework. A case study on the design of on-demand heating, ventilation, and air conditioning (HVAC) systems is presented to demonstrate the use of PBD.

In the era of Internet of Things, crowd counting, which estimates the number of people within a region, becomes the underpinning for many emerging applications, such as occupancy estimation in smart building and queuing management and product placement in shopping center. Existing vision based crowd counting schemes require favorable lighting conditions and also raise privacy concerns. RF based approaches rely on specialized sensors and require users to carry RF devices. Thus, an accurate, reliable and non-intrusive crowd counting scheme is still desired. In this paper, we propose FreeCount, a device-free crowd counting scheme that is able to precisely estimate the number of people within a region using only commodity WiFi routers. To this end, the channel state information (CSI) data in PHY layer is obtained directly by upgrading the router's software. We propose an information theory based feature selection scheme to select the most representative features that are sensitive to human motion. To build a classifier that is robust to temporal and environmental disparities, we adopt transfer kernel learning, which minimizes the difference between the source and target distributions in the reproducing kernel Hilbert space, is adopted to process the real-time CSI feature data. Experiments were conducted in moderate sized rooms and the results demonstrated that FreeCount is able to accurately estimate the number of people with 96% crowd counting accuracy consistently over temporal and environmental variation.

Human activity recognition is becoming the vital underpinning for a myriad of emerging applications in the field of human-computer interaction, mobile computing, and smart grid. Besides the utilization of up-to-date sensing techniques, modern activity recognition systems also require a machine learning (ML) algorithm that leverages the sensory data for identification purposes. In view of the unique characteristics of the measurement data and the ML challenges thereof, we propose a non-intrusive human activity recognition system that only uses existing commodity WiFi routers. The core of our system is a novel multiple kernel representation learning (MKRL) framework that automatically extracts and combines informative patterns from the Channel State Information (CSI) measurements. The MKRL firstly learns a kernel string representation from time, frequency, wavelet, and shape domains with an efficient greedy algorithm. Then it performs information fusion from diverse perspectives based on multi-view kernel learning. Moreover, different stages of MKRL can be seamlessly integrated into a multiple kernel learning framework to build up a robust and comprehensive activity classifier. Extensive experiments are conducted in typical indoor environments and the experimental results demonstrate that the proposed system outperforms existing methods and achieves a 98\% activity recognition accuracy.

Existing human activity recognition approaches require either the deployment of extra infrastructure or the cooperation of occupants to carry dedicated devices, which are expensive, intrusive and inconvenient for pervasive implementation. In this paper, we propose SmartSense, a device-free human activity recognition system based on a novel machine learning algorithm with existing commercial off-the-shelf (COTS) WiFi routers. By exploiting the prevalence of existing WiFi infrastructure in buildings, we developed a novel OpenWrt based firmware for COTS WiFi routers to collect the CSI measurements from regular data frames. To identify different human activities, an automatic kernel representation learning method, namely auto-HSRL, is established to selection informative Hilbert space patterns from time, frequency, wavelet, and shape domains. A new information fusion tool based on multi-view kernel learning is proposed to combine the representations extracted from diverse perspectives and build up a robust and comprehensive activity classifier. Extensive experiments were conducted in an office and the experimental results demonstrate that SmartSense outperforms existing methods and achieves a 98% activity recognition accuracy.

Detecting dangerous riding behaviors is of great importance to improve bicycling safety. Existing bike safety precautionary measures rely on dedicated infrastructures that incur high installation costs. In this work, we propose BikeMate, a ubiquitous bicycling behavior monitoring system with smartphones. BikeMate invokes smartphone sensors to infer dangerous riding behaviors including lane weaving, standing pedalling and wrong-way riding. For easy adoption, BikeMate leverages transfer learning to reduce the overhead of training models for different users, and applies crowdsourcing to infer legal riding directions without prior knowledge. Experiments with 12 participants show that BikeMate achieves an overall accuracy of 86.8% for lane weaving and standing pedalling detection, and yields a detection accuracy of 90% for wrong-way riding using crowdsourced GPS traces.

Predicting blood glucose dynamics is vital for people to take preventive measures in time against health risks. Previous efforts adopt handcrafted features and design prediction models for each person, which result in low accuracy due to ineffective feature representation and the limited training data. This work proposes MT-LSTM, a multi-time-series deep LSTM model for accurate and efficient blood glucose concentration prediction. MT-LSTM automatically learns feature representations and temporal dependencies of blood glucose dynamics by jointly sharing data among multiple users and utilizes an individual learning layer for personalized prediction. Evaluations on 112 users demonstrate that MT-LSTM significant outperform conventional predictive regression models.

Accurate Location Based Service (LBS) is one of the fundamental but crucial services in the era of Internet of Things (IoT). WiFi fingerprinting-based Indoor Positioning System (IPS) has become the most promising solution for indoor LBS. However, the offline calibrated received signal strength (RSS) radio map is unable to provide consistent LBS with high localization accuracy under various environmental dynamics. To address this issue, we propose TKL-WinSMS as a systematic strategy, which is able to realize robust and adaptive indoor localization in dynamic indoor environments. We developed a WiFi-based Non-intrusive Sensing and Monitoring System (WinSMS) that enables WiFi routers as online reference points by extracting real-time RSS readings among them. With these online data and labeled source data from the offline calibrated radio map, we further combine the RSS readings from target mobile devices as unlabeled target data, to design a robust localization model using an emerging transfer learning algorithm, namely transfer kernel learning (TKL). It is able to learn a domain-invariant kernel by directly matching the source and target distributions in the reproducing kernel Hilbert space instead of the raw noisy signal space. The resultant kernel can be used as input for the SVR training procedure. In this manner, the trained localization model can inherit the information from online phase to adaptively enhance the offline calibrated radio map. Extensive experiments were conducted and demonstrated that the proposed TKL- WinSMS is able to improve the localization accuracy by at least 26% compared with existing solutions under various environmental interferences.

WiFi fingerprinting-based indoor positioning system (IPS) has become the most promising solution for indoor localization. However, there are two major drawbacks that hamper its large-scale implementation. First, an offline site survey process is required which is extremely time-consuming and labor-intensive. Second, the RSS fingerprint database built offline is vulnerable to environmental dynamics. To address these issues comprehensively, in this paper, we propose WinIPS, a WiFi-based non-intrusive IPS that enables automatic online radio map construction and adaptation, aiming for calibration-free indoor localization. WinIPS can capture data packets transmitted in existing WiFi traffic and extract the RSS and MAC addresses of both WiFi access points (APs) and mobile devices in a non-intrusive manner. APs can be used as online reference points for radio map construction. A novel Gaussian process regression model is proposed to approximate the non-uniform RSS distribution of an indoor environment. Extensive experiments were conducted, which demonstrated that WinIPS outperforms existing solutions in terms of both RSS estimation accuracy and localization accuracy.

In the era of Internet of Things, WiFi fingerprinting based indoor positioning system (IPS) has been recognized as the most promising IPS for indoor location-based service. Fingerprinting-based algorithms critically rely on a fingerprint database built from machine learning methods, and extreme machine learning (ELM) is preferred for its fast training speed. However, traditional WiFi based IPS usually requires a central server to collect and process data, which is tremendously vulnerable to server breakdown and communication link failure. To address this issue, we propose Consensus-based Parallel ELM (CPELM) to enhance the robustness by distributing the data on different computation nodes. Specifically, each node keeps updating the corresponding terms in the ELM regression equation as a weighted average of those from neighboring nodes based on the distributed consensus iterative scheme. Upon the agreement of the regression equation within the network, the output weight of ELM can be calculated on some nodes and propagated to other nodes. Extensive simulation with real data has demonstrated that CPELM is able to produce same level of localization accuracy as centralized ELM without incurring additional computational cost, and in the meanwhile provides more robustness to the entire IPS in case of server breakdown and link failures.

Pedestrian dead reckoning (PDR) is a promising complementary technique to balance the requirements on both accuracy and costs in outdoor and indoor positioning systems. In this paper, we propose a unified framework to comprehensively tackle the three sub problems involved in PDR, including step detection and counting, heading estimation and step length estimation, based on sequentially rotating the device (reference) frame to the Earth (reference) frame through sensor fusion. To be specific, a robust step detection and counting algorithm is devised according to vertical angular velocities and turns out to be tolerant of various smartphone placements; then, a zero velocity update (ZUPT) based algorithm is leveraged to calibrate the measurements in the Earth frame; on these grounds, the heading and step length are further estimated by exploiting the cyclic features of walking. A thorough and extensive experimental analysis is conducted and confirms the effectiveness and advantages of the proposed PDR framework as well as the corresponding algorithms.

Existing WiFi fingerprinting-based Indoor Positioning System (IPS) suffers from the vulnerability of environmental dynamics. To address this issue, we propose TKL-WinSMS as a systematic strategy, which is able to realize robust and adaptive localization in dynamic indoor environments. We developed a WiFi-based Non-intrusive Sensing and Monitoring System (WinSMS) that enables COTS WiFi routers as online reference points by extracting real-time RSS readings among them. With these online data and labeled source data from the offline calibrated radio map, we further combine the RSS readings from target mobile devices as unlabeled target data, to design a robust localization model using an emerging transfer learning algorithm, namely transfer kernel learning (TKL). It can learn a domain-invariant kernel by directly matching the source and target distributions in the reproducing kernel Hilbert space instead of the raw noisy signal space. By leveraging the resultant kernel as input for the SVR training, the trained localization model can inherit the information from online phase to adaptively enhance the offline calibrated radio map. Extensive experimental results verify the superiority of TKL-WinSMS in terms of localization accuracy compared with existing solutions in dynamic indoor environments.

The explosive proliferation of mobile devices and the popularity of social networks have spurred extensive demands on Location Based Services (LBSs) in recent decades. The IEEE 802.11 (WiFi) based Indoor Positioning Systems (IPSs) are gaining popularity because of the wide and ubiquitous availability of WiFi infrastructures in indoor environments. Most of IPSs are adopting the fingerprinting approach to mitigate pervasive indoor multipath effects. However, the heterogeneity of mobile devices significantly degrades the localization performance of the fingerprinting approach. In this paper, we apply the Procrustes analysis method to transform the WiFi received signal strengths (RSSs) to a new type of standard location fingerprints which are tolerant of the heterogeneity of various devices. Then, a robust indoor positioning algorithm based on the standardized location fingerprints and the weighted k nearest neighbor (WKN-N) method is proposed. Extensive experiments are carried out and show that the standardized location fingerprints and the proposed positioning system address the device heterogeneity issue satisfactorily.

Accurate occupancy information of indoor environments is one of the key prerequisites for many pervasive and context-aware services, e.g. smart building/home systems. Some of the existing occupancy inference systems can achieve impressive accuracy, but they either require labour-intensive calibration phases, or need to install bespoke hardware such as CCTV cameras, which are privacy-intrusive by default. In this paper, we present the design and implementation of a practical end-to-end occupancy inference system, which requires minimum user effort, and is able to infer room-level occupancy accurately with commodity WiFi infrastructure. Depending on the needs of different occupancy information subscribers, our system is flexible enough to switch between snapshot estimation mode and continuous inference mode, to trade estimation accuracy for delay and communication cost. We evaluate the system on a hardware testbed deployed in a 600m2workspace with 25 occupants for 6 weeks. Experimental results show that the proposed system significantly outperforms competing systems in both inference accuracy and robustness.

The increasing demands of location-based services have spurred the rapid development of indoor positioning system and indoor localization system interchangeably (IPSs). However, the performance of IPSs suffers from noisy measurements. In this paper, two kinds of robust extreme learning machines (RELMs), corresponding to the close-to-mean constraint, and the small-residual constraint, have been proposed to address the issue of noisy measurements in IPSs. Based on whether the feature mapping in extreme learning machine is explicit, we respectively provide random-hidden-nodes and kernelized formulations of RELMs by second order cone programming. Furthermore, the computation of the covariance in feature space is discussed. Simulations and real-world indoor localization experiments are extensively carried out and the results demonstrate that the proposed algorithms can not only improve the accuracy and repeatability, but also reduce the deviation and worst case error of IPSs compared with other baseline algorithms.

Indoor positioning system (IPS) has become one of the most attractive research fields due to the increasing demands on location-based services (LBSs) in indoor environments. Various IPSs have been developed under different circumstances, and most of them adopt the fingerprinting technique to mitigate pervasive indoor multipath effects. However, the performance of the fingerprinting technique severely suffers from device heterogeneity existing across commercial off-the-shelf mobile devices (e.g., smart phones, tablet computers, etc.) and indoor environmental changes (e.g., the number, distribution and activities of people, the placement of furniture, etc.). In this paper, we transform the received signal strength (RSS) to a standardized location fingerprint based on the Procrustes analysis, and introduce a similarity metric, termed signal tendency index (STI), for matching standardized fingerprints. An analysis of the capability of the proposed STI to handle device heterogeneity and environmental changes is presented. We further develop a robust and precise IPS by integrating the merits of both the STI and weighted extreme learning machine (WELM). Finally, extensive experiments are carried out and a performance comparison with existing solutions verifies the superiority of the proposed IPS in terms of robustness to device heterogeneity.

This article describes a method for indoor positioning of human-carried active Radio Frequency Identification (RFID) tags based on the Sampling Importance Resampling (SIR) particle filtering algorithm. To use particle filtering methods, it is necessary to furnish statistical state transition and observation distributions. The state transition distribution is obstacle-aware and sampled from a precomputed accessibility map. The observation distribution is empirically determined by ground truth RSS measurements while moving the RFID tags along a known trajectory. From this data, we generate estimates of the sensor measurement distributions, grouped by distance, between the tag and sensor. A grid of 24 sensors is deployed in an office environment, measuring Received Signal Strength (RSS) from the tags, and a multithreaded program is written to implement the method. We discuss the accuracy of the method using a verification data set collected during a field-operational test.

Extreme learning machine (ELM) as an emergent technology has shown its good performance in regression applications as well as in large dataset classification applications. It has been broadly embedded in many applications due to its fast speed of computation and accuracy. How to make good use of machine learning techniques in Indoor Positioning System (IPS) is a hot research topic in recent years. Some existing IPSs have already adopted ELM, but it suffers from signal variation and environmental dynamics in indoor settings. In this paper, extreme learning machine with dead zone (DZ-ELM) is proposed to address this problem. The consistency of this approach should be applied is studied. Simulations are also conducted to compare the performance of DZ-ELM and ELM. Lastly, real-world experimental results show that the proposed algorithm can not only provide higher accuracy but also improve the repeatability of IPSs.

We present results from a set of experiments in this pilot study to investigate the causal influence of user activity on various environmental parameters monitored by occupant-carried multi-purpose sensors. Hypotheses with respect to each type of measurements are verified, including temperature, humidity, and light level collected during eight typical activities: sitting in lab / cubicle, indoor walking / running, resting after physical activity, climbing stairs, taking elevators, and outdoor walking. Our main contribution is the development of features for activity and location recognition based on environmental measurements, which exploit location- and activity-specific characteristics and capture the trends resulted from the underlying physiological process. The features are statistically shown to have good separability and are also information-rich. Fusing environmental sensing together with acceleration is shown to achieve classification accuracy as high as 99.13%. For building applications, this study motivates a sensor fusion paradigm for learning individualized activity, location, and environmental preferences for energy management and user comfort.

In recent years, applying RFID technology to develop an Indoor Positioning System (IPS) has become a hot research topic. The most prominent advantage of active RFID IPS comes from its unique identification of different objects in indoor environment. However, certain drawbacks of existing RFID IPSs, such as high cost of RFID readers and active tags, as well as heavy dependence on the density of reference tags to provide the location based service, largely limit the applications of active RFID IPS. In order to overcome these drawbacks, we develop a cost-efficient RFID IPS by using cheaper active RFID tags, sensors and reader. In addition, one localization algorithm: integrated Weighted Path Loss (WPL) - Extreme Learning Machine (ELM) which combines the fast estimation of WPL and the high localization accuracy of ELM is proposed. According to the algorithm, an indoor environment is divided into small zones firstly and an ELM model is developed for each zone during the offline phase. During the online phase, the WPL approach is used to determine the zone of the target primarily, then the ELM model of that zone is deployed to provide the final estimated location of the target. Based on our experimental result, this integrated algorithm provides a higher localization efficiency and accuracy than existing approaches.