RFID has revolutionized item-level intelligence in IoT ecosystems, yet static localization with passive tags remains challenging due to multipath interference inherent in RF signals. We present LuxTag, the first system to enable visible light-based sensing and localization using standard, commercial RFID tags by transforming them into ambient light sensors. Our key insight leverages the discovery that photon-induced leakage currents in passive RFID ICs modulate their persistence time (i.e., the duration a tag remains operational after RF excitation ceases) proportional to ambient illuminance. LuxTag introduces two innovations: (i) a first-principles model characterizing how ambient light alters tag persistence time, enabling battery-free light sensing without hardware modifications; (ii) a differential measurement technique and zero-shot calibration method to isolate light effects and autonomously derive tag parameters, ensuring robust and accurate static localization system using COTS RFID infrastructure. Extensive experiments demonstrate that LuxTag achieves a mean light intensity error of 3.6 lux and 60.7% improvement over state-of-the-art static RFID localization. By synergizing the ubiquity of RFID with the multipath resilience of optical sensing, LuxTag opens new avenues for static RFID localization in smart warehouses, retails, and beyond.

While gaining significant attention for device-free applications, Wi-Fi sensing still faces challenges in differentiating multiple targets; this stems from the design priorities of Wi-Fi systems that prioritize coverage and stability over sensing diversity. Existing proposals that either expand bandwidth or increase antennas to enhance sensing diversity can be confined by the limited access to Wi-Fi firm/hardware. To this end, we propose Mirror-Fi, a novel Wi-Fi sensing system that improves sensing diversity without modifying Wi-Fi firm/hardware. Exploiting the reconfigurability of metasurfaces, Mirror-Fi augments beamforming with spatially significant features, facilitating the construction of exclusive sensing signal links for individual targets. We innovate in an encoding scheme that equips each metasurface with a distinct phase coding sequence to mark link uniqueness. We then train a deep neural model to leverage prior coding sequences for decomposing non-linearly superimposed channel samples into mutually independent channels; it removes the need for complex channel matrix parameter estimation and mitigates hardware-related offsets inherent to Wi-Fi. Extensive evaluations demonstrate that, with a sufficient number of auto-configured metasurfaces, Mirror-Fi successfully achieves multi-target sensing.

Millimeter-wave (mmWave) radar offers significant potential for fine-grained sensing, yet transitioning from controlled laboratory environments to dynamic real-world applications remains challenging. Existing methods face a dichotomy: real-time point clouds sacrifice crucial signal details needed for sophisticated tasks, whereas information-rich raw data sensing imposes prohibitive transmission and computation overheads, often limiting analysis to offline settings and hindering real-time viability. To this end, we present CoSense, areal-timeedge-end collaborative sensing system built on commodity mmWave radar (end) and edge intelligence. We first introduce a novel dual-stream data acquisition mechanism via realizing radar driver-level interfaces, enabling simultaneous transmission of point clouds and raw data. To bridge the fidelity-latency trade-off, we implement an adaptive transmission strategy via firmware modifications, selectively forwarding raw data segments (corresponding to regions of interest identified in the point cloud) for detailed fine-grained analysis, while continuously delivering point clouds for low-latency coarse-grained sensing and control loops. Furthermore, we incorporate closed-loop feedback beamforming, dynamically steering the radar beam based on real-time tracking to counteract motion-induced misalignment and enhance signal fidelity. Extensive evaluations under dynamic conditions demonstrate that CoSense successfully achieves real-time fine-grained sensing with high fidelity and manageable overhead.

Speech enhancement is crucial for reliable communication in noisy environments. However, the lack of a priori knowledge about target speech characteristics in conventional systems often leads to erroneous extraction of interfering speech as desired signals during noise suppression, significantly compromising system performance. Recently, researchers have proposed to use the side-channel signal as an assist to denoise noisy speech. This paper proposes BoneSE, a multimodal speech enhancement approach using bone conduction. The basic idea of BoneSE is to perceive the bone-conducted sound with an IMU sensor embedded in the Commercial Off-The-Shelf (COTS) earphone and then leverage the correlations between the bone conduction signal and audio signal for speech enhancement. However, the lack of high-frequency components in the IMU modality brings data imbalance and hinders data fusion. To address this challenge, we explore and model the relationship between multimodal signals and design a fusion module according to the time and frequency correlation. Moreover, to balance fast processing and denoising performance, we propose bone conduction-based noise level metrics to measure the noise level. To accommodate different noise levels, we propose an adaptive model selection approach based on reinforcement learning to select the proper denoising model, thereby optimizing the latency. Experiments on two datasets show that the proposed method performs favorably against state-of-the-art methods and can enhance speech in low SNR scenarios.

Nowadays, video stream analysis has a wide range of applications, and videos captured by mobile devices are often used for such analyses. Stable videos provide a better foundation for downstream analysis. Current stabilization methods usually estimate camera motion using feature points, but moving objects can be mistaken for camera movement. Moreover, the loss of depth can cause image distortion during video smoothing. To address above challenges, we propose a multi-modal video stabilization method for dynamic scenes, assisted by the gyroscope embedded in the mobile device. Firstly, we notice that the rotation of feature points extracted from trees, buildings or other static objects, should be consistent with camera rotation, while feature points extracted from cars and other dynamic objects have completely different rotation. Hence, we cluster feature points and calculate the rotation for each cluster, then compare their rotation with gyroscope rotation to distinguish static feature points. Secondly, as it takes much time to calculate the absolute depth of feature points, we suggest leveraging gyro measurements to estimate the relative depth of feature points in a light-weight way. Furthermore, we can classify feature points by relative depth, allowing us to apply different smoothing techniques to points at different depths, thereby enhancing the video stabilization effect. Experimental results show that, compared to traditional multimodal methods, we can improve video stability (SSIM value) by 47.8% and increase processing speed by 37%, effectively enhancing the quality of shaky videos in dynamic and loss of depth scenes while achieving greater efficiency.

As the low-altitude economy expands, Unmanned Aerial Vehicles (UAVs) have become essential across a variety of applications. However, non-cooperative UAVs threaten personal privacy and public safety, especially when carrying dangerous payloads. Current UAV detection methods primarily rely on computer vision, which often fails in severe weather conditions. This paper introduces mmUAVsense, a mmWave-based system designed for detecting and identifying UAVs in complex environments. In addition to detecting UAVs, mmUAVsense can also estimate the rotational speed of their propellers, which offers valuable information for assessing the potential payload. A key challenge in detecting UAVs is their small Radar Cross Section (RCS), which makes them easily masked by the environment, especially when flying near large objects. To address this issue, we propose a Moving Target Indicator (MTI) to calculate the time-spatial difference, which helps suppress static environmental clutter and make the dynamic reflection from the UAV more noticeable. To differentiate UAVs from other flying objects, we exploit the unique harmonic feature in the UAV’s Doppler spectrum, caused by the rapid spinning of its propellers, to identify UAVs among other airborne objects. To estimate the UAV’s rotational speed, we analyze the relationship between harmonic features and propeller speed, and propose to use Chirp-Z Transformation (CZT) to accurately extract the frequency from the spectrum. We have implemented a system prototype in various outdoor environments, and extensive experiments demonstrate that our system can identify UAVs with over 95% accuracy, with an average rotational speed error of just 3.8%. Based on the existing UAV model, our rotary-sensing-based system can sense the UAV’s extra load weight with only 8 grams of average error.

Human pose recognition is a crucial task in millimeter-wave point cloud processing, playing an important role in scenarios such as autonomous driving, human-computer interaction, and medical monitoring. Currently, most approaches for recognizing human poses from millimeter-wave point clouds rely on deep learning methods. However, due to the impact of multipath effects in wireless signals, millimeter-wave point clouds not only contain human pose information but also include environmental noise. The neural network extracts features from all point clouds equally, which leads to errors in human pose recognition caused by environmental noise interference. To address this limitation, we propose TSDA, a novel Temporal-Spatial Data Augmentation method specifically tailored for human pose recognition in millimeter-wave point clouds. TSDA leverages the temporal consistency and spatial continuity of human pose point clouds to distinguish the real human point clouds from environmental noise point clouds. It quantifies the probability of a point cloud belonging to the human pose target through confidence measures, which can be used to augment the raw point clouds. Additionally, we introduce a prototype system based on cross-attention mechanisms to validate the impact of data augmentation on pose recognition performance. Experimental results show that, compared to the state-of-the-art deep learning-based methods, TSDA achieves an average reduction of 1.9cm in human pose recognition error.

Recent advances in sign language research have benefited from CNN-based backbones, which are primarily transferred from traditional computer vision tasks (\eg object detection, image recognition). However, these CNN-based backbones usually excel at extracting features like contours and texture, but may struggle with capturing sign-related features. To capture such sign-related features, SignGraph model extracts the cross-region sign features by building the Local Sign Graph (LSG) module and the Temporal Sign Graph (TSG) module. However, we emphasize that although capturing cross-region dependencies can improve sign language performance, it may degrade the representation quality of local regions. To mitigate this, we introduce MixSignGraph, which represents sign sequences as a group of mixed graphs for feature extraction. Specifically, besides the LSG module and TSG module that model the intra-frame and inter-frame cross-regions features, we design a simple yet effective Hierarchical Sign Graph (HSG) module, which enhances local region representations following the extraction of cross-region features, by aggregating the same-region features from different-granularity feature maps of a frame, \ie to boost discriminative local features. In addition, to further improve the performance of gloss-free sign language task, we propose a simple yet counter-intuitive Text-based CTC Pre-training (TCTC) method, which generates pseudo gloss labels from text sequences for model pre-training. Extensive experiments conducted on the current five sign language datasets demonstrate that MixSignGraph surpasses the most current models on multiple sign language tasks across several datasets, without relying on any additional cues. Code and models are available at: \href{https://github.com/gswycf/SignLanguage}{\textcolor{blue}{https://github.com/gswycf/SignLanguage}}.

Image stitching is often affected by obstructions such as pedestrians and vehicles. Traditional methods often ignore obstructions or retain them, leading to artifacts. They also cause edge distortion due to fixed-perspective stitching. To address above issues, this poster presents IIS, an IMU-assisted Image Stitching method. It uses IMU data for perspective pre-alignment, projecting images to an intermediate perspective to reduce edge distortion. It simultaneously removes obstructions and mitigating their impact. A weighted target scene stitching strategy is integrated to further enhance stitching quality. IIS effectively reduces artifacts and maintains high computational efficiency.

Person re-identification (Re-ID) aims to retrieve images of the same individual from a gallery captured by disjoint camera views. A major challenge lies in learning robust and discriminative representations under varying human poses and cluttered backgrounds. Recent approaches based on Generative Adversarial Networks (GANs) attempt to mitigate these issues by augmenting training data via pose or style transfer. However, despite generating visually plausible samples, such methods often introduce redundant or low-quality data, which can hinder feature learning, slow down convergence, and lead to overfitting. In this paper, we propose PGQE, a Pose-Guided Query Enhancement framework that synthesizes pose-normalized query images during inference, thereby avoiding the drawbacks of GAN-based data augmentation in training. Motivated by the observation that queries with cleaner backgrounds and canonical poses yield better matching performance, PGQE leverages high-quality gallery samples to extract target poses and guide image generation. To ensure the quality and discriminability of the generated queries, we impose two constraints: identity consistency with the source image and pose consistency with the extracted target pose. Extensive experiments on Market-1501, CUHK03, and MSMT17 demonstrate that PGQE significantly improves ReID accuracy and outperforms several state-of-the-art methods.

With the rapid development of smart health care, accurate heart rate variability (HRV) estimation for the early detection of diseases has become a hot research topic. Advanced work uses the wireless signal to estimate the heartbeat in a contact-free way, which usually cannot separate multiple users or work in a dynamic environment. In this article, we propose a lightweight heartbeat-sensing method based on RFID tag pairs, which focuses on HRV extraction in a more general sensing scenario. Based on the tag-pair design, we build a novel heartbeat and respiration model to describe the signal relationship between the two tags from the time and space domains. Based on the model, we propose a Calibrated Temporal-Spatial IQ-Shaping-based signal cancellation algorithm to cancel the respiration and extract the heartbeat. To remove the interference in dynamic measurement, we build an IQ-based signal model via a Principal Component Analysis-based interference estimation. To reduce the statistical error in HRV extraction, we further design a neural network to predict the HRV index. We have implemented a system prototype in a real environment with COTS RFID devices. Extensive experiments show that our system can achieve a median RMSSD error of 7.51 ms, which satisfies the medical demand in HRV measurement.

International audience

With the rise of intelligent systems like assisted driving and robotics, all-weather target identification and 3D localization systems have become crucial for reliable obstacle avoidance and navigation. However, vision-based methods struggle to provide accurate target locations under low light or bad weather. Radar-based solutions like mmWave radar and LiDAR are robust but hindered by high costs and challenges in recognizing target identities at scale. In this paper, we propose alow-cost, all-weather target identification and 3D localization systembased onLED-tags, which system can address the needs of intelligent systems for obstacle avoidance in complex environments. We explore the backscatter communication of LED devices and design adual-modal LED-Tag, which includes two features: a backscatter RF signal detectable by RF devices and visual light spot information detectable by cameras, both sharing the same ID. To enhance the limited backscatter capability, we propose amulti-branch parallel modelthat enhances the signal strength using beamforming synthesis and achannel adjustment mechanismto improve robustness in complex environments, ensuring accurate 3D localization. For multi-target identification, we design an LED-tag encoding system, assigning each tag a unique encoding sequence. Each target's identity can be recognized with our customizedID decoding method, which leverages prior information and time-domain sampling characteristics. Extensive experimental results show that the backscatter communication and target detection range of LED-tags can reach15m. Moreover, the system achieves anaverage localization error of 7.3cm within a 5m range, demonstrating the system's excellent performance in terms of practicality and accuracy.

Using multiple Unmanned Aerial Vehicles (UAVs) in video surveillance greatly enhances real-time monitoring of large areas. However, images captured by UAVs are separate and limited in view, making stitching crucial for a comprehensive perspective. Current methods combine sensor and visual modalities for video stitching but face challenges in robustness and real-time performance. Environmental disturbances increase errors in sensor and visual data, reducing accuracy and stability, while single-frame stitching incurs high computational costs and slow speeds. To address these issues, we propose SkyEye , a real-time video stitching method for UAV surveillance systems based on multi-modal perception. Specifically, we design a mutual verification method to assess the quality of visual and inertial data. Frames with higher confidence are prioritized for precise stitching. To reduce computational costs, we employ a reference frame scheme that reuses the perspective transformation and feature points of the reference frame for subsequent frames. Meanwhile, we design a video stitching framework based on a per-frame parallel unit to speed up the algorithm execution. We have implemented a prototype of SkyEye and carried out extensive evaluations. Experiment results show that SkyEye outperforms state-of-the-art methods, improving speed by 66.1% and accuracy by 28.2%.

Despite the recent success of sign language research, the widely adopted CNN-based backbones are mainly migrated from other computer vision tasks, in which the contours and texture of objects are crucial for identifying objects. They usually treat sign frames as grids and may fail to capture effective cross-region features. In fact, sign language tasks need to focus on the correlation of different regions in one frame and the interaction of different regions among adjacent frames for identifying a sign sequence. In this paper, we propose to represent a sign sequence as graphs and introduce a simple yet effective graph-based sign language processing architecture named SignGraph, to extract crossregion features at the graph level. SignGraph consists of two basic modules: Local Sign Graph (LSG) module for learning the correlation of intra-frame cross-region features in one frame and Temporal Sign Graph (TSG) module for tracking the interaction of inter-frame cross-region features among adjacent frames. With LSG and TSG, we build our model in a multiscale manner to ensure that the representation of nodes can capture cross-region features at different granularities. Extensive experiments on current public sign language datasets demonstrate the superiority of our SignGraph model. Our model achieves very competitive performances with the SOTA model, while not using any extra cues. Code and models are available at: https://github.com/gswycf/SignGraph.

Nowadays, object detection and 3D tracking have become key technologies for intelligent system or robot navigation to realize automatic obstacle avoidance and target detection, especially in low-light and night vision scenarios. In this paper, we explore the backscattering capability of LEDs and implement a multi-modal tag LED-tag to realize object detection and 3D tracking. Our basic idea is to utilize the fact that feeding modulation signals to an LED-tag can generate both RF and visual features. We fuse the depth of field information perceived from the RF domain and the pixel coordinates obtained from the visual domain to derive a 3D position by matching the decoded ID. In the RF domain, the depth of field is acquired through ultra-wideband channel measurements and estimated phase. In the visual domain, the pixel coordinate in the XOY coordinates can be extract from the image and mapped into 2D spatial coordinates. To address the limited backscatter capability of the LED-tag, we propose a multiple parallel branch model to increase backscatter paths for amplifying the LED-tag's backscattering intensity. Additionally, we propose a decoding ID scheme that utilizes a priori knowledge and repetitive samples to restore the IDs whose encoding frequency is higher than four times the sampling rate. We have implemented a prototype system and evaluated its performance in real-world environments. Extensive experimental results show that LED-tag can backscatter RF signals ranging up to 15m. Besides, the system achieves an average position error of 8cm within the range of 3m.

Ultra-high precision motion sensing leveraging computer vision (CV) is a key technology in many high-precision AR/VR applications such as precise industrial manufacture and image-guided surgery, yet conventional CV can be challenged by moiré-based sensing mechanism, thanks to moiré pattern's high sensitivity to six degrees of freedom (6-DoF) pose changes. Unfortunately, existing moiré-based solutions, in their infancy, cannot deal with complicated curvilinear moiré patterns caused by various perspective angles. In this paper, we propose a generalized moiré-based mechanism, MoiréVision, towards practical adoptions; it relies on high-frequency gratings as visual marker to help extract the fine-grained feature points for ultra-high precision motion sensing. As the foundation of general moiré-based sensing, we propose a formulation to characterize "uncontrolled" curvilinear moiré patterns in practical scenarios. To deal with the problem of moiré feature interference in practice, we propose a Gabor-based algorithm to separate overlapped curvilinear moiré patterns from two dimensions. Furthermore, to extract fine-grained feature points for high-precision motion sensing, we propose a bending function-based model and a resolution-enhanced strategy to reconstruct detailed texture of moiré markers and extract moiré feature points at sub-pixel level. Extensive experimental results show that MoiréVision greatly enhances the usability and generalizability of moiré-based sensing systems in real-world applications.

Detection and assessment of micro vibrations are crucial tasks in both industrial settings and daily life. However, vibration sensors attached to the target vibrator may introduce potential resonance, and wireless detection methods suffer from severe multipath interference. Fortunately, moiré-based sensing methods have gained recognition in recent years due to their ability to perceive micro motion changes. In this paper, we propose MoiréVib, a micro-vibration detection solution based on moiré patterns for dynamic and high-frequency environments. We attach a printed marker with periodic gratings to the surface of vibration devices to generate moiré patterns, which can amplify micro vibrations due to their low-frequency magnification effect. However, moiré pattern's changes caused by micro vibrations are often overwhelmed by random pixel-level noises, and the limited frame rate of the camera fails to capture high-frequency moiré features. To deal with these problems, we propose a spectrum-based method to refine and enhance the dynamic and micro moiré features. Additionally, we propose a dual-frame-rate-based fusion mechanism to realize high-frequency reconstruction of moiré features. Extensive experimental results show that MoiréVib can realize a median amplitude detection error of 4.37 μm and achieve frequency detection up to 300Hz with a frame rate range of 10~30 fps.

In the realm of AR applications and particularly camera-to-screen interactions, camera tracking stands as a crucial technology. However, the ever-increasing demand for tracking accuracy makes it essential to explore a six-degrees of freedom (6-DoF) tracking technology with ultra-high precision to facilitate micro-motion sensing. In this paper, we propose a novel sensing method MoiréTracker to achieve camera’s 6-DoF pose tracking with ultra-high precision. MoiréTracker outputs camera’s continuous 3-DoF trajectory and 3-DoF posture changes according to the captured moiré patterns, which can be produced by the superposition of camera’s Color Filter Array (CFA) and the projection of screen raster on the CFA plane. Thanks to moiré pattern’s high sensitivity to 6-DoF motions, we characterize the relationship between moiré features and camera’s micro pose changes, so as to realize the continuous 6-DoF pose tracking for camera with ultra-high precision. Moreover, our proposal involves a thumbnail-based method aimed at expanding the working range of MoiréTracker, enabling the pervasive camera-to-screen interactions. We implement a prototype system and evaluate its performance in real-world environments. Extensive experiment results show that MoiréTracker achieves the average trajectory error of 1.20 cm and the posture error of 1.07°.

With the rising of demands for novel Human-Computer Interaction (HCI) approaches in the 3D space, a number of intelligent approaches have been proposed to achieve the HCI by tracking the translation and rotation of the target devices. In this paper, we propose to realize a light-weight, battery-free, 3D motion tracking solution by leveraging a spinning linearly polarized antenna to track a passive RFID tag array. Instead of using the fixed antennas, which can only receive stable signal in some specific environments due to the unpredictable multipath effect, we propose to mitigate the multipath effect and the ambient interference by continuously spinning a linearly polarized antenna, and then extract the most distinctive features based on the optimal reading conditions of the spinning antenna. In particular, because the phase variation around the matching direction is more stable while the RSSI variation around the mismatching direction is more distinctive, we leverage such matching/mismatching property of the linearly polarized antenna to extract the most distinctive features for motion tracking. To depict the property, we build a theoretical model to explain the RSSI and the phase variation of the RFID tag along with the spinning of the antenna, and further extend the model from a single RFID tag to an RFID tag array. Based on the model, we can extract the distinctive RSSI features for the rotation tracking and the stable phase features for the translation tracking. Moreover, to tackle the low rate of feature extraction due to the spinning of antenna, we further propose to enhance the unstable phase features based on the overall trend of other tags with interpolation, such that the sampling rate can be efficiently improved. Finally, we propose a LSTM (Long Short Term Memory)-based network to track the 3D motion based on the signal features extracted based on the polarization model. The experimental results show that our system can achieve an average error of 10.45 cm in the translation tracking, and an average error of$6.02^\circ$in the rotation tracking in the 3D space.

Nowadays, to realize the intelligent manufacturing in Industrial Internet of Things (IIoT) scenarios, novel approaches in computer vision are in great demand to tackle the new challenges in IIoT environment. These approaches, which we callIndustrial Vision, are expected to offer customized solutions for intelligent manufacturing in an accurate, time efficient and robust manner. In this paper, we propose a novel approach to industrial vision, calledEdge-Eye, to rectify the edge deviation automatically for Irradiated Cross-linked Polyethylene Foam (IXPE) production with millimeter-level accuracy. IXPE has been one of the most commonly used materials in industry. During the production process of IXPE sheets, their edges need keep aligned strictly, otherwise, they could quickly get out of the border of the rolling plate and cause the huge economic loss. We deploy a commercial camera with mobile edge node in front of the IXPE sheet to continuously detect and rectify the edge deviation. Particularly, to handle the complex production environment when extracting the edge of IXPE sheet, we deploy a pair of reference bars with high-contrast colors to efficiently differentiate the sheet edge from the background. Then, we propose aBi-direction Edge Tracking methodto perform the edge detection from both vertical and horizontal aspects. To realize the rectification using mobile edge nodes with limited computing resources, we reduce the cost of computation by extracting theMinimized Region of Interest, i.e., the edge area overlapped with the higher contrast reference bar on both sides. We further design a negative feedback control system with multi-stage feedback regulation mechanism, keeping the edge deviation withinmillimeter-level. We implementedEdge-Eyeon the ARM64 platform and performed evaluation in the practical IXPE production process. The experimental results show thatEdge-Eyeachieves the average accuracy of 5 mm for the edge deviation rectification, with the average latency of 200 ms for edge deviation detection. During the process of 20-month real deployment for 36 production lines, 66 manpower per day (90% of the overall manpower) has been saved, and the utilization rate of IXPE material increases from 87% to 94%.

Speech is a natural communication way between people and a good way for human-computer interaction. However, speech with audible voices often faces the following problems, e.g., being affected by surrounding noises, breaking the quiet environment, leaking privacy, etc. Therefore, silent speech was proposed, especially lip reading, which aims to recognize speech content based on lip movements. In this paper, we utilize inaudible acoustic signals generated from mobile device to sense and recognize lip movements for lip reading. Considering the lack of public dataset in acoustic-based lip reading, we propose and release a large-scale lip-reading dataset${\sf LIPCMD}$with 30000 acoustic-based recordings. To advance the further research in lip reading, we provide benchmark evaluation on${\sf LIPCMD}$, while using traditional machine learning solutions and recent deep learning approaches. To recognize weak acoustic signals as words for lip reading, we propose a self distillation based approachLipReader, which distills the probability distribution and attention map in convolutional neural network itself for better classification. Finally, we implementLipReaderon smartphone and evaluate it on${\sf LIPCMD}$dataset as well as under complex scenarios. Experimental results show thatLipReadercan achieve a good recognition accuracy for lip reading, i.e., 91.58%, while outperforming baseline solutions and existing work.

In industrial production, the orientation of facility components can indicate whether the facility is on a regular operating track. For example, when a component gets loose, the orientation variation of the component would exceed the normal range. A common approach for orientation measurement is to attach an inertial measurement unit (IMU) to the target device. However, the IMU requires additional power maintenance. This article presents LightGyro, a cheap and efficient batteryless scheme to measure the orientation, in which we attach a reflective film to the target device and use a camera to capture the light spot on the reflective film. The basic idea of LightGyro is to extract the light spots in the captured frame and use their pixel coordinates to infer the orientation. It is difficult to recognize a single light spot, because the spot lacks distinctive features. To solve the problem, we switch light sources on and off to regulate the appearance of light spots and utilize frame subtraction to extract light spots. The depth of field of light spot is lost in the process of camera projection, which is necessary for the orientation measurement. To address the issue, we propose a light array-based reflection model to extract the depth of field from the relative positions of multiple light spots. To the best of our knowledge, this is the first work to utilize reflection to measure orientation. Experiment results show that the orientation error of LightGyro decreases with the increasing length of the reflection route and the orientation error can achieve less than 1 ˆ .

Handheld devices such as smartphones and cameras often produce shaky videos due to reasons such as hand tremors and movement. Current video stabilization methods mainly estimate camera motion based on feature points, such as corner points, and then smooth the motion to obtain a stabilized video. However, in reality, most videos are shot in dynamic scenes, where some feature points are extracted from moving objects. They seriously affect the estimation of camera motion, and result in unsatisfactory video stabilization. In this paper, we propose a multi-modal-based video stabilization method for dynamic scenes using video frames and gyroscope. The basic idea is to eliminate dynamic feature as their estimated rotation significantly differs from the actual rotation obtained from the gyroscope data. The experimental results show that our method effectively enhances the quality of shaky videos in dynamic scenes.

With the rapid development of smart factories, it is necessary to provide lightweight and real-time working condition monitoring for remote facility sensors to reduce labor costs and ensure safe operation. In the high-voltage power grid scenario, it is crucial to detect the working state of the electric knife-edge switch, i.e., the switch's rotation angle in 3D space, which controls the on-off state of the high-voltage lines. However, existing solutions are mainly in a contact way, which is easy to cause accidents in high risk scenarios such as high-voltage transmission lines. In this paper, we propose EdgeMonitor, a contactless monitoring system for knife-edge switches, which can provide robust and real-time working state detection based on lightweight edge devices. EdgeMonitor includes a lightweight edge device, a monocular camera, and an inertial sensor. Specifically, we propose a lightweight two-stage strategy first to extract the switch's feature information, then perform continuous state tracking for the switch's rotation angle in 3D space. Experiment results show that EdgeMonitor achieves the knife-edge switches detection error within 2.5° with a latency of less than 50ms.

There are two problems that widely exist in current end-to-end sign language processing architecture. One is the CTC spike phenomenon which weakens the visual representational ability in Continuous Sign Language Recognition (CSLR). The other one is the exposure bias problem which leads to the accumulation of translation errors during inference in Sign Language Translation (SLT). In this paper, we tackle these issues by introducing contrast learning, aiming to enhance both visual-level feature representation and semantic-level error tolerance. Specifically, to alleviate CTC spike phenomenon and enhance visual-level representation, we design a visual contrastive loss by minimizing visual feature distance between different augmented samples of frames in one sign video, so that the model can further explore features by utilizing numerous unlabeled frames in an unsupervised way. To alleviate exposure bias problem and improve semantic-level error tolerance, we design a semantic contrastive loss by re-inputting the predicted sentence into semantic module and comparing features of ground-truth sequence and predicted sequence, for exposing model to its own mistakes. Besides, we propose two new metrics, i.e., Blank Rate and Consecutive Wrong Word Rate to directly reflect our improvement on the two problems. Extensive experimental results on current sign language datasets demonstrate the effectiveness of our approach, which achieves state-of-the-art performance.

With the proliferation of voice assistants, microphone-based speech recognition technology usually cannot achieve good performance in the situation of multiple sound sources and ambient noises. In this paper, we propose a novel mmWave-based solution to perform speech recognition to tackle the issues of multiple sound sources and ambient noises, by precisely extracting the multi-modal features from lip motion and vocal-cords vibration from the single channel of mmWave. We propose a difference-based method for feature extraction of lip motion to suppress the dynamic interference from body motion and head motion. We propose a speech detection method based on cross-validation of lip motion and vocal-cords vibration so as to avoid wasting computing resources on nonspeaking activities. We propose a multi-modal fusion framework for speech recognition by fusing the signal features from lip motion and vocal-cords vibration with the attention mechanism. We implemented a prototype system and evaluated the performance in real test-beds. Experiment results show that the average speech recognition accuracy is 92.8% in realistic environments.

With the popularity of online meetings equipped with speakers, voice privacy security has drawn increasing attention because eavesdropping on the speakers can quickly obtain sensitive information. In this paper, we propose mmEavesdropper, a mmWave based eavesdropping system, which focuses on augmenting the micro-vibration signal via theoretical models for voice recovery. Particularly, to augment the receiving signal of the target vibration, we propose to use beam-forming to facilitate the directional augmentation by suppressing other orientations and use Chirp-Z transform to facilitate the distance augmentation by increasing the range resolution compared with traditional FFT. To augment the vibration signal in the IQ plane, we build a theoretical model to analyze the distortion and propose a segmentation-based fitting method to calibrate the vibration signal. To augment the spectrum for sound recovery, we propose to combine multiple channels and leverage an encoder-decoder based neural network to reconstruct the spectrogram for voice recovery. We perform extensive experiments on mmEavesdropper and the results show that mmEavesdropper can reach the accuracy of 93% on digit and letter recognition. Moreover, mmEavesdropper can reconstruct the voice with an average SNR of 5dB and peak SNR of 17dB.

To provide instant communication for hearing-impaired people, it is essential to achieve real-time sign language processing anytime anywhere. Therefore, in this paper, we propose a Region-aware Temporal Graph based neural Network (RTG-Net), aiming to achieve real-time Sign Language Recognition (SLR) and Translation (SLT) on edge devices. To reduce the computation overhead, we first construct a shallow graph convolution network to reduce model size by decreasing model depth. Besides, we apply structural re-parameterization to fuse the convolutional layer, batch normalization layer and all branches to simplify model complexity by reducing model width. To achieve the high performance in sign language processing as well, we extract key regions based on keypoints in skeleton from each frame, and design a region-aware temporal graph to combine key regions and full frame for feature representation. In RTG-Net, we design a multi-stage training strategy to optimize keypoint selection, SLR and SLT step by step. Experimental results demonstrate that RTG-Net achieves comparable performance with existing methods in SLR or SLT, while greatly reducing the computation overhead and achieving real-time sign language processing on edge devices. Our code is available at https://github.com/SignLanguageCode/realtimeSLRT.

With the popularity of smartwatches, users can access private information stored in the device by simply touching the watch screen. However, smartwatches also expose users to the risk of information leakage because they lack proper authentication schemes. This paper proposes PalmEcho, a multimodal authentication scheme for smartwatches. The basic idea of PalmEcho is to capture the vibration and the sound generated from user’s beating gestures, and then fuse multimodal signals to extract unique features for user authentication. However, signals of beating gestures are short in the time domain, which makes it hard to extract effective features. To address this challenge, our work reveals that the spectral energy distribution of the generated sound is unique to each user and provides rich information for authentication. Moreover, conventional classification networks require large amounts of user data for training, which is not convenient in the user authentication scenario. To address this challenge, we design a prototypical network called BeatNet, which allows users to register with a few samples. Experimental results show that PalmEcho can reach an average F1-score of 94%.

Keystroke snooping is an effective way to steal sensitive information from the victims. Recent research on acoustic emanation-based techniques has greatly improved the accessibility by non-professional adversaries. However, these approaches either require multiple smartphones or require specific placement of the smartphone relative to the keyboards, which tremendously restricts the application scenarios. In this article, we propose UltraSnoop, a training-free, transferable, and placement-agnostic scheme that manages to infer user’s input using a single smartphone placed within the range covered by a microphone and speaker. The innovation of Ultrasnoop is that we propose an ultrasonic anchor-keystroke positioning method and a Mel Frequency Cepstrum Coefficients clustering algorithm, synthesis of which could infer the relative position between the smartphone and the keyboard. Along with the keystroke time difference of arrival, our method could infer the keystrokes and even gradually improve the accuracy as the snooping proceeds. Our real-world experiments show that UltraSnoop could achieve more than 85% top-3 snooping accuracy when the smartphone is placed within the range of 30–60 cm from the keyboard.

We consider the multiuser detection (MUD) problem, i.e., how to separate and decode colliding data streams, in the uplink of massive Machine Type Communications (mMTC) at millimeter wave (mmWave). Operating on factor-graphs by passing messages, the sum-product algorithm and its variants are widely applied in many other scenarios. However, in this paper, we find that their performance in mMTC at mmWave could be dramatically degraded due to the ill-conditioned MUD channel gain matrix and the existence of enormous short cycles in their corresponding factor-graphs, which are caused by the limited scattering of mmWave and the sharing of a same codebook for error correction among densely located user equipments. Assuming LDPC codes are used for error correction, we further propose a novel sum-product based approach to dealing with the MUD problem in mMTC at mmWave. It first leverages the propagation characteristics of mmWave to optimize the factor-graph for MUD by removing short cycles based on node-split and node-contraction, and then takes a dynamic-programming based method to approximate the messages passing on the resulted factor-graph, which can achieve a higher decoding accuracy. Extensive simulation results show that our approach outperforms the state-of-the-art sum-product based approaches significantly.

Nowadays, authentication systems are usually required to provide continuous, contactless, and non-intrusive services. In this paper, we proposeRF-Badge, a vital sign-based authentication scheme on human subjects to meet the above requirements by using RFID technology. We consider two biometric features with individual diversity to characterize the vital sign of users, including themovement effectfrom respiration and thereflection effectfrom organs, especially the heart. To derive the movement effect from respiration, we build a phase-based geometric model to restore the fine-grained badge moving trace as the feature. To derive the reflection effect from human internal organs, we extract the reflection signal from the original signal and generate the spectrum as the feature. Besides, to deal with the feature deviation in different physical conditions of users, we propose a multi-condition network (MCNet) to further guarantee the generalization of RF-Badge. We implement a prototype system and evaluate the performance in real environments. The experiment results show that our system achieves the average false positive rate (FPR) of 3.9 percent and false negative rate (FNR) of 3.3 percent for continuous authentication within four signal cycles.

Passive RFID technology is widely used in user authentication and access control. We propose RF-Rhythm, a secure and usable two-factor RFID authentication system with strong resilience to lost/stolen/cloned RFID cards. In RF-Rhythm, each legitimate user performs a sequence of taps on his/her RFID card according to a self-chosen secret melody. Such rhythmic taps can induce phase changes in the backscattered signals, which the RFID reader can detect to recover the user’s tapping rhythm. In addition to verifying the RFID card’s identification information as usual, the backend server compares the extracted tapping rhythm with what it acquires in the user enrollment phase. The user passes authentication checks if and only if both verifications succeed. We also propose a novel phase-hopping protocol in which the RFID reader emits Continuous Wave (CW) with random phases for extracting the user’s secret tapping rhythm. Our protocol can prevent a capable adversary from extracting and then replaying a legitimate tapping rhythm from sniffed RFID signals. Comprehensive user experiments confirm the high security and usability of RF-Rhythm with false-positive and false-negative rates close to zero.

In the context of Industrial Internet of Things (IIoT), RFID technologies have been widely applied to locate or track tagged objects for achieving item-level intelligence. However, prior localization work encounters two main issues. First, the phase measurement usually contains physical deviation. Existing localization work generally takes the physical center of an RFID antenna as its phase center, which is a key factor in improving localization accuracy but actually different from the physical center in practice. Second, the non-linear localization model is likely to be too complex to run on edge nodes with limited computing resources. In this paper, we present a LInear localizatiON solution, called LION, to perform the phase calibration for antennas with no need for the complex computation nor strong limitations. Specifically, we provide a novel lightweight model to pinpoint the actual antenna position quickly and accurately. Compared to previous localization methods, we reduce the intersection of circles or hyperbolas into radical lines, which greatly reduces the computation cost while guaranteeing the high accuracy. Further, to adapt to the complex environment with various ambient noise and multi-path effect, we leverage the weighted least square method to determine the optimal position. Moreover, we propose an adaptive parameter selection scheme to automatically choose optimal parameters for localization. In this way, LION is able to perform the accurate localization robustly. We implement LION using commercial RFID devices, and evaluate its performance extensively. Experimental results show the necessity of phase calibration as well as the high time efficiency of LION, e.g., the average accuracy improves by 6× and 2.1× for 2D and 3D localization, and the average time consuming is 0.02s and 1.8s for 2D and 3D cases.

As a key component of most machines, the status of the rotation shaft is a crucial issue in the factories, which affects both the industrial safety and the product quality. Tracking the rotation angle can efficiently monitor the status of the rotation shaft, but traditional solutions either rely on the specialized sensors, suffering from intrusive transformation, or use the computer vision-based solutions, suffering from poor light conditions. In this paper, we present a non-contacting low-cost angle tracking solution, RF-Protractor, to track the rotation shaft based on the surrounding RFID tags. Particularly, instead of directly attaching the tags to the shaft, which may lead to serious miss reading problems due to metal interference, we deploy the tags beside the shaft and leverage the polarization effect of the reflection signal from the shaft for angle tracking. To improve the polarization effect, we exploit the linear polarization feature by using the linear shaft turntable or placing a light aluminum foil on the shaft turntable, which requires no transformation of the shaft. We firstly build a polarization model to quantify the relationship between the rotation angle and the reflection signal. To extract the accurate reflection signal, we then propose to combine the signals of multiple tags to cancel the reflection effect and then estimate the environment-related parameter to calibrate the model. Finally, we propose to leverage both the power trend and the IQ signal to estimate the rotation direction and the rotation angle. We have implemented a real system and the extensive experiments in the real environment confirm the effectiveness of RF-Protractor, which achieves an average error of about 3.1° in angle tracking.

Voice assistant has been widely used for human-computer interaction and automatic meeting minutes. However, for multiple sound sources, the performance of speech recognition in voice assistant decreases dramatically. Therefore, it is crucial to separate multiple voices efficiently for an effective voice assistant application in multi-user scenarios. In this paper, we present a novel voice separation system using a 2D microphone array in multiple sound source scenarios. Specifically, we propose a spatial filtering-based method to iteratively estimate the Angle of Arrival (AoA) of each sound source and separate the voice signals with adaptive beamforming. We use BeamForming-based cross-Correlation (BF-Correlation) to accurately assess the performance of beamforming and automatically optimize the voice separation in the iterative framework. Different from cross-correlation, BF-Correlation further performs cross-correlation among the after-beamforming voice signals processed with each linear microphone array. In this way, the mutual interference from voice signals out of the specified direction can be effectively suppressed or mitigated via the spatial filtering technique. We implement a prototype system and evaluate its performance in real environments. Experimental results show that the average AoA error is 1.4 degree and the average ratio of automatic speech recognition accuracy is 90.2% in the presence of three sound sources.

Irradiated Cross-linked Polyethylene Foam (IXPE) has been one of the most commonly used materials in industry. During the production process of IXPE sheets, their edges need keep aligned strictly, otherwise, they could quickly get out of the border of the rolling plate and cause the huge economic loss. In this paper, we propose a camera-enabled approach, called Edge-Eye, to rectify the edge deviation automatically for IXPE production with millimeter-level accuracy. We deploy a commercial camera with mobile edge node in front of the IXPE sheet to continuously detect and rectify the edge deviation. Particularly, to handle the complex production en-vironment when extracting the edge of IXPE sheet, we deploy a pair of reference bars with high-contrast colors to efficiently differ-entiate the sheet edge from the background. Then, we propose a Bi-direction Edge Tracking method to perform the edge detection from both vertical and horizontal aspects. To realize the rectification using mobile edge nodes with limited computing resources, we reduce the cost of computation by extracting the Minimized Region of Interest, i.e., the edge area overlapped with the higher contrast reference bar on both sides. We further design a negative feedback control system with multi-stage feedback regulation mechanism, keeping the edge deviation within millimeter-level. We implemented Edge-Eye on the ARM64 platform and performed evaluation in the practical IXPE production process. The experimental results show that Edge-Eye achieves the average accuracy of 5mm for the edge deviation rectification, with the average latency of 200ms for edge deviation detection. During the process of 20-month real deployment for 36 production lines, 66 manpower per day (90% of the overall manpower) has been saved, and the utilization rate of IXPE material increases from 87% to 94%.

Camera tracking has become a key technology for various application scenarios, especially for AR-based camera-to-screen interaction. Demand for subtle motion detection in camera tracking makes it essential to explore the six degrees of freedom (6-DoF) pose detection with ultra-high precision. In this paper, we propose a novel sensing method MoiréPose to achieve ultra-high precision on the camera's 6-DoF pose estimation. The purpose of MoiréPose is to derive the camera's 3-DoF position and 3-DoF posture relative to the screen according to the captured moiré pattern, which is produced by the superposition of the camera's Color Filter Array (CFA) and the screen raster projected onto the CFA layer. Based on moiré pattern's high sensitivity to 6-DoF pose movement and robustness to the environmental interference in the frequency domain, we propose a spectrogram-based method to realize the camera's 6-DoF detection with ultra-high precision. Moreover, we propose a thumbnail-based method to effectively extend the working range of MoiréPose, so as to realize pervasive camera-to-screen interaction. We have implemented a prototype system and evaluate the performance in real-world environments. Extensive experiment results show that MoiréPose achieves an average position error of 7.5mm and an overall posture error of 1.66°.

In industrial production, the orientation of facility is a powerful indicator to verify whether the facility is in a normal operating track. In this paper, we present LightGyro, a cheap and efficient batteryless scheme to measure the facility orientation, it leverages the orientation amplification effect of reflection to improve the measuring accuracy to one degree. LightGyro system is composed of low-cost camera, batteryless reflective film and LEDs. In the working process of LightGyro, we attach a reflective film to the target and let it reflect the light from LEDs to the camera. Then the LightGyro would extract the LED-related spots in the captured frame and restore the reflection route to measure the orientation. To extract the LED-related spots from complicated background automatically, we propose to leverage the affine transformation to search for the topology of multiple spots which is related to the deployed LED array. To address the dimension missing issue caused by camera projection and restore the reflection route, we propose a light array-based reflection model to extract the missing dimension from relative positions of multiple spots. To the best of our knowledge, this is the first work to utilize light reflection to measure orientation. Our experiments show that the average accuracy of LightGyro achieves less than 2◦. When the reflective film is far from the camera, the mean error is less than 1◦.

Multi-subject video analysis is one of the most important problems in the field of visual perception for human activity recognition on multiple subjects nowadays. However, multi-subject video analysis is difficult to achieve real-time performance at the edge due to the limited resources of edge devices and the high complexity of the Convolutional Neural Networks (CNN) model used in this task. The common processing method is to upload the video data to the cloud. However, due to the influence of network bandwidth, the transmission time is not fixed, and the latency cannot be guaranteed. Thus, statically deployed model configurations cannot meet some dynamically changing scenarios. To address these challenges, in this paper, we propose an edge-cloud collaboration processing system for multi-subject video stream analysis, which can dynamically configure and optimize the related configurations according to specific scenarios. Specifically, we provide an adaptive configuration optimization solution based on context awareness for edge devices with limited resources such that multi-subject video stream analysis can be processed completely at the edge. For other complex scenarios, we propose an edge-cloud collaboration method to achieve task segmentation and collaboration to meet the performance requirements of the complex scenarios. Experimental results show that our method can achieve an average accuracy of 91.3% and the latency of less than 78ms with arbitrary runtime state.

Mobile and wearable devices have become more and more popular. However, the tiny touch screen leads to inefficient interaction with these devices, especially for text input. In this article, we proposeDynaKey, which allows people to type on a virtual keyboard printed on a piece of article or drawn on a desk, for inputting text into a head-mounted camera device (e.g., smart glasses). By using the built-in camera and gyroscope, we capture image frames during typing and detect possible head movements, then track keys, detect fingertips, and locate keystrokes. To track the changes of keys’ coordinates in images caused by natural head (i.e., camera) movements, we introduce perspective transformation to transform keys’ coordinates among different frames. To detect and locate keystrokes, we utilize the variation of fingertip’s coordinates across multiple frames to detect possible keystrokes for localization. To reduce the time cost, we combine gyroscope and camera to adaptively track the keys, and introduce a series of optimizations, such as keypoint detection, frame skipping, multithread processing, etc. Finally, we implement DynaKey on Android-powered devices. The extensive experimental results show that our system can efficiently track and locate the keystrokes in real time. Specifically, the average tracking deviation of the keyboard layout is less than 3 pixels and the Intersection over Union (IoU) of a key in two consecutive images is above 93%. The average keystroke localization accuracy reaches 95.5%.

With the rising of demands for novel human-computer interaction approaches in the 2D plane, a number of intelligent devices come into being. For example, Microsoft Surface Dial supports simple clicks and rotations for the interaction with computer. However, these approaches are dedicated devices, and they might require batteries or have limited functions. In this paper, we propose RF-Dial to realize a light-weight, battery-free and functional 2D human-computer interaction solution via commercial off-the-shelf (COTS) passive RFID tags. What RF-Dial shines is that it can easily turn an ordinary object, e.g., a board eraser, into an intelligent interaction device. By deploying a tag array on the side face of the object together with a dipole tag on the top face, RF-Dial cannot only track the rigid motion of the object but also detect the touch gesture of a user on the surface of the object, including translation, rotation, click, press and hold, and swipe. To do the motion tracking, RF-Dial builds a phase-based model that captures the translation and the rotation of the tagged object simultaneously, by jointly exploiting the information of phase variations and the topology of the tag array. To detect the touch gesture, RF-Dial builds an RSSI-based model that uses the impact of the touching finger on the tag antenna’s impedance to estimate the touch position in real time, which is robust to environmental factors like position or orientation. We implemented a prototype of RF-Dial with commodity RFID devices. Extensive experiments show that RF-Dial achieves an accurate rigid motion tracking, with a small error of 0.6cm for the translation tracking, and a small error of 1.9 degrees for the rotation estimation. Besides, RF-Dial can also detect the touch gesture accurately, as the 90 percent of touch position errors are less than 2.09mm.

Among all the road accidents, speeding is the most deadly factor. To reduce speeding, it is essential to devise efficient schemes for ubiquitous speed monitoring. Traditional approaches either suffers from using special equipment(e.g., radar speed gun) or special deployment(e.g., position-fixed cameras). In this article, we propose SpeedTalker, a mobile phone-based approach to perform speed detection on automobiles. By leveraging the built-in microphones and camera from the mobile phone, SpeedTalker estimates the automobile speed by passively sensing the acoustic and image signals. We propose an integrated solution to effectively estimate the automobile’s speed based on COTS devices, and provide a platform for every pedestrian to help report the speeding event of automobiles. Specifically, we use the time difference of arrivals (TDOA) model based on acoustic signals to figure out the candidate trajectories of automobile, and use the pin-hole model based on image frames to figure out the vertical distance between the user’s position and the automobile’s trajectory, thus to estimate the unique trajectory. Combined with the time stamp of the trajectory, the automobile speed can be estimated. Besides, we propose a method to effectively mitigate the influence of the movement jitters of mobile phone. We implemented a system prototype for SpeedTalker and estimated the automobile speed with high accuracy. Experiment results show that in the scenario of single automobile, SpeedTalker can achieve an average estimation error of 6.1 percent compared to radar speed guns. In the scenario of multiple automobiles, SpeedTalker can achieve an average estimation error of 9.8 percent, which is acceptable for usage.

This work enhances the machine-to-human communication between electronic toll collection (ETC) systems and drivers by providing an AM broadcast service to deployed ETC systems. This study is the first to show that ultra-high radio frequency identification signals can be received by an AM radio receiver due to the presence of the nonlinearity effect in the AM receiver. Such a phenomenon allows the development of a previously infeasible cross-technology and cross-frequency communication, called Tagcaster, which converts an ETC reader to an AM station for broadcasting short messages (e.g., charged-fees and traffic forecast) to drivers at tollbooths. The key innovation in this work is the engineering of Tagcaster over off-the-shelf ETC systems using shadow carrier and baseband whitening without the need for hardware nor firmware changes. This feature allows zero-cost rapid deployment in the existing ETC infrastructure. Two prototypes of Tagcaster are designed, implemented, and evaluated over four general and five vehicle-mounted AM receivers (e.g., Toyota, Audi, and Jetta). Experiments reveal that Tagcaster can provide good-quality (PESQ>2) and stable AM broadcasting service with a 30 m coverage range. Tagcaster remarkably improves user experience at ETC stations, and two-thirds of volunteer drivers rate it with a score of 4+ out of 5.

Nowadays, detecting and evaluating the internal structure of packages becomes a crucial task for logistics systems to guarantee reliability and security. However, prior solutions such as X-ray diffraction and WiFi-based detection are not suitable for this purpose. X-ray-based methods usually require manual analysis or image processing algorithms with high complexity, while WiFi-based solutions may fail to detect complex structures due to the significant error of the RF-signal features. In this article, we propose RF-Detector, a low-cost RFID solution for performing three-dimensional (3D) structure detection of items contained in the packages, including the item orientations and relative locations. We thoroughly investigate a brand-new sensing model for RFID-based 3D structure detection, i.e., revolving scanning. We propose not only the fundamental revolving model but also a novel calibration method for the undesired deployments. We have implemented a prototype system to evaluate the performance of RF-Detector. Extensive evaluations in real settings show the effectiveness of RF-Detector, achieving very high accuracy of the internal 3D structure detection.

Gait rehabilitation is a common method of postoperative recovery after the user sustains an injury or disability. However, traditional gait rehabilitations are usually performed under the supervision of rehabilitation specialists, which implies that the patients cannot receive adequate gait assessment anytime and anywhere. In this article, we propose GaitTracker, a novel system to remotely and continuously perform gait monitoring and analysis by three-dimensional (3D) skeletal tracking in a wearable approach. Specifically, this system consists of four Inertial Measurement Units (IMU), which are attached on the shanks and thighs of the human body. According to the measurements from these IMUs, we can obtain the motion signals of lower limbs during gait rehabilitation. By adaptively synchronizing coordinate systems of different IMUs and building the geometric model of lower limbs, the exact gait movements can be reconstructed, and gait parameters can be extracted without any prior knowledge. GaitTracker offers three key features: (1) a unified 3D skeletal model to depict the precise gait movement and parameters in 3D space, (2) a coordinate system synchronization scheme to perform space synchronization over all the IMU sensors, and (3) an automatic estimation method for the user-specific geometric parameters. In this way, GaitTracker is able to accurately perform 3D skeletal tracking of lower limbs for gait analysis, such as evaluating the gait symmetry and the gait parameters including the swing/stance time. We implemented GaitTracker and evaluated its performance in real applications. The experimental results show that, the average error for skeleton angle estimation, joint displacement estimation, and gait parameter estimation are 3∘, 2.3%, and 3%, respectively, outperforming the state of the art.

As an essential communication way for deaf-mutes, sign languages are expressed by human actions. To distinguish human actions for sign language understanding, the skeleton which contains position information of human pose can provide an important cue, since different actions usually correspond to different poses/skeletons. However, skeleton has not been fully studied for Sign Language Translation (SLT), especially for end-to-end SLT. Therefore, in this paper, we propose a novel end-to-end Skeleton-Aware neural Network (SANet) for video-based SLT. Specifically, to achieve end-to-end SLT, we design a self-contained branch for skeleton extraction. To efficiently guide the feature extraction from video with skeletons, we concatenate the skeleton channel and RGB channels of each frame for feature extraction. To distinguish the importance of clips, we construct a skeleton-based Graph Convolutional Network (GCN) for feature scaling, i.e., giving importance weight for each clip. The scaled features of each clip are then sent to a decoder module to generate spoken language. In our SANet, a joint training strategy is designed to optimize skeleton extraction and sign language translation jointly. Experimental results on two large scale SLT datasets demonstrate the effectiveness of our approach, which outperforms the state-of-the-art methods. Our code is available at https://github.com/SignLanguageCode/SANet.

Recent research advancement of wireless sensing technology has made device-free interaction in the WiFi-enabled IoT environment possible. Although gesture-based interaction with such a smart environment greatly improves usability, it also introduces many security problems, such as shoulder surfing attacks. By spoofing the gestures of legitimate users, the attacker could easily access private information or services and cause even worse consequences. A secure interaction mechanism for this environment is required to prevent attackers without compromising the usability, while the limited recognition ability and low robustness of WiFi sensing make this target extremely challenging. To this end, we propose a secure interaction mechanism called secure interaction via WiFi Signal (SiWi), which provides the ability to resist shoulder surfing attacks without compromising the usability by using just WiFi signals. SiWi innovates in a concurrent interaction/authentication framework with only three elemental gestures (push, swing, and wave) and four types of identity-related imperceptible/hidden features (time distribution, direction, angle, and distance). HMM and Fresnel model-based algorithms are used to recognize the gestures and extract hidden features robustly and efficiently. Extensive experiments in a real implemented system were conducted to investigate the effectiveness of the proposed secure interaction system. The results show that our system can achieve an average accuracy of 93% to identify legitimate users and 97% to resist the spoofer.

Currently, the logistic industry has introduced 3D reconstruction to monitor the package placement in the warehouse. Previous 3D reconstruction solutions mainly utilize computer vision or sensor-based methods, which are restricted to the line-of-sight or the battery life. Therefore, we propose a passive RFID-based solution, called RF-3DScan, to perform 3D reconstruction on tagged packages, including the package orientation and the package stacking. The basic idea is that a moving antenna can obtain RF-signals from the tags attached on packages with the 1D linear mobile scanning. Through extracting phase differences to build angle profiles for each tag, RF-3DScan derives their relative positions, further determines the package orientation and the coarse-grained package stacking. By simply performing the 2D scanning, RF-3DScan can provide the fine-grained package stacking determination. We implement a prototype system of RF-3DScan and evaluate its performance in real settings. Our experiment results show that RF-3DScan can achieve about 92.5 percent identification accuracy of the bottom face, and an average error about 4.08° of thethe rotation angle. For the package stacking, 1D scanning can achieve the comparable performance in comparison with 2D scanning.

Due to the sudden movement during the camera shoot, the videos retrieved from the hand-held mobile devices often suffer from undesired frame jitters, leading to the loss of video quality. In this paper, we present a video stabilization solution in mobile devices via inertial-visual state tracking. Specifically, during the video shoot, we use the gyroscope to estimate therotationof camera, and use the structure-from-motion among the image frames to estimate thetranslationof camera. We build a camera projection model by considering the rotation and translation of the camera, and the camera motion model to depict the relationship between the inertial-visual state and the camera's 3D motion. By fusing the inertial measurement (IMU)-based method and the computer vision (CV)-based method, our solution is robust to the fast movement and violent jitters, moreover, it greatly reduces the computation overhead in video stabilization. In comparison to the IMU-based solution, our solution can estimate the translation in a more accurate manner, since we use the feature point pairs in adjacent image frames, rather than the error-prone accelerometers, to estimate the translation. In comparison to the CV-based solution, our solution can estimate the translation with less number of feature point pairs, since the number of undetermined degrees of freedom in the 3D motion directly reduces from 6 to 3. We implemented a prototype system on smart glasses and smart phones, and evaluated the performance under real scenarios, i.e., the human subjects used mobile devices to shoot videos while they were walking, climbing or riding. The experiment results show that our solution achieves 32 percent better performance than the state-of-art solutions in regard to video stabilization. Moreover, the average processing time latency is 32.6ms, which is lower than the conventional inter-frame time interval, i.e., 33ms, and thus meets the real-time requirement for online processing.

In traditional device‐to‐device (D2D) communication based on wireless channel, identity authentication and spontaneous secure connections between smart devices are essential requirements. In this paper, we propose an imitation‐resistant secure pairing framework including authentication and key generation for smart devices, by shaking these devices together. Based on the data collected by multiple sensors of smart devices, these devices can authenticate each other and generate a unique and consistent symmetric key only when they are shaken together. We have conducted comprehensive experimental study on shaking various devices. Based on this study, we have listed several novel observations and extracted important clues for key generation. We propose a series of innovative technologies to generate highly unique and completely randomized symmetric keys among these devices, and the generation process is robust to noise and protects privacy. Our experimental results show that our system can accurately and efficiently generate keys and authenticate each other.

Nowadays, detecting and evaluating the internal structure of packages becomes a crucial task for logistics systems to guarantee the reliability and security. However, prior solutions such as X-ray diffraction and WiFi-based detection are not suitable for this purpose. X-ray-based methods usually require manual analysis or image processing algorithms with high complexity, while WiFi-based solutions may fail to detect complex structures due to the significant error of the RF-signal features. In this paper, we propose RF-Detector, a low-cost RFID solution for performing 3D structure detection of items contained in the packages, including the item orientations and relative locations. We thoroughly investigate a brand-new sensing model for RFID-based 3D structure detection, i.e., revolving scanning. We propose not only the fundamental revolving model but also a novel calibration method towards the undesired deployment. We have implemented a prototype system to evaluate the performance of RF-Detector. Extensive evaluations in real settings show the effectiveness of RF-Detector, achieving very high accuracy of the internal 3D structure detection.

This work enhances the machine-to-human communication between electronic toll collection (ETC) systems and drivers by providing an AM broadcast service to deployed ETC systems. This study is the first to show that ultra-high radio frequency identification signals can be received by an AM radio receiver due to the presence of the nonlinearity effect in the AM receiver. Such a phenomenon allows the development of a previously infeasible cross-technology and cross-frequency communication, called Tagcaster, which converts an ETC reader to an AM station for broadcasting short messages (e.g., charged- fees and traffic forecast) to drivers at tollbooths. The key innovation in this work is the engineering of Tagcaster over off-the-shelf ETC systems using shadow carrier and baseband whitening without the need for hardware nor firmware changes. This feature allows zero-cost rapid deployment in existing ETC infrastructure. Two prototypes of Tagcaster are designed, implemented and evaluated over four general and five vehicle-mounted AM receivers (e.g., Toyota, Audi, and Jetta). Experiments reveal that Tagcaster can provide good-quality (PESQ> 2) and stable AM broadcasting service with a 30 m coverage range. Tagcaster remarkably improves user experience at ETC stations and two- thirds volunteer drivers rate it with a score of 4+ out of 5.

Federated learning is a very promising machine learning paradigm where a large number of clients cooperatively train a global model using their respective local data. In this paper, we consider the application of federated learning in wireless networks featuring uplink multiuser multiple-input and multiple-output (MU-MIMO), and aim at optimizing the communication efficiency during the aggregation of client-side updates by exploiting the inherent superposition of radio frequency (RF) signals. We propose a novel approach named Physical-Layer Arithmetic (PhyArith), where the clients encode their local updates into aligned digital sequences which are converted into RF signals for sending to the server simultaneously, and the server directly recovers the exact summation of these updates as required from the superimposed RF signal by employing a customized sum-product algorithm. PhyArith is compatible with commodity devices due to the use of full digital operation in both the client-side encoding and the server-side decoding processes, and can also be integrated with other updates compression based acceleration techniques. Simulation results show that PhyArith further improves the communication efficiency by 1.5 to 3 times for training LeNet-5, compared with solutions only applying updates compression.

Passive RFID technology is widely used in user authentication and access control. We propose RF-Rhythm, a secure and usable two-factor RFID authentication system with strong resilience to lost/stolen/cloned RFID cards. In RF-Rhythm, each legitimate user performs a sequence of taps on his/her RFID card according to a self-chosen secret melody. Such rhythmic taps can induce phase changes in the backscattered signals, which the RFID reader can detect to recover the user’s tapping rhythm. In addition to verifying the RFID card’s identification information as usual, the backend server compares the extracted tapping rhythm with what it acquires in the user enrollment phase. The user passes authentication checks if and only if both verifications succeed. We also propose a novel phase-hopping protocol in which the RFID reader emits Continuous Wave (CW) with random phases for extracting the user’s secret tapping rhythm. Our protocol can prevent a capable adversary from extracting and then replaying a legitimate tapping rhythm from sniffed RFID signals. Comprehensive user experiments confirm the high security and usability of RF-Rhythm with false-positive and false-negative rates close to zero.

Cellular network operators deploy base stations with a high density to ensure radio signal coverage for 4G/5G networks. While users enjoy the high-speed connection provided by cellular networks, an adversary could exploit the dense cellular deployment to detect nearby human movements and even recognize keystroke movements of a victim by passively listening to the CRS broadcast from base stations. To demonstrate this, we develop SpiderMon, the first attempt to perform passive continuous keystroke monitoring using the signal transmitted by commercial cellular base stations. Our experimental results show that SpiderMon can detect keystroke movements at a distance of 15 meters and can recover a 6-digits PIN input with a success rate of more than 51% within ten trials when the victim is behind the wall.

In typical scenarios for indoor localization and tracking, it is essential to accurately track the pedestrians when they are crossing the connections of different spaces. In this paper, we propose a magnet barcode scanning-based solution for indoor pedestrian tracking. We assemble multiple magnet bars into magnet arrays as a unique magnet barcode, and deploy different magnet barcodes at different connections to label them. We embed an inertial measurement unit (IMU) into the pedestrian`s shoes. When the pedestrian crosses these connections, the magnetometer from the IMU scans the magnet barcode and recognize its corresponding ID. In this way, indoor pedestrian tracking can be regarded as a process of continuously scanning different magnet barcodes. By performing correlation analysis on these barcodes, the trace of pedestrian can be effectively depicted in the indoor map. To build a unique magnet barcode based on the magnet bar arrays, we provide an optimized structure for building the magnet barcode. To tackle the diversities of the pedestrian's gait traces in identifying the magnet barcode, we provide a generalized model based on the space axis for magnet barcode identification. As far as we know, this is the first work to use the magnet bar array to construct the magnet barcode for indoor pedestrian tracking. The real experiment results show that our system can achieve an average accuracy of 88.9% in identifying the magnet barcodes and an average accuracy of 93.1 % for indoor pedestrian tracking.

Nowadays, gesture recognition plays a more and more important role in human-computer interaction. In this regard, contact sensors or computer vision have made some progress, but they also have shortcomings in portability or privacy. In this work, we propose a gesture recognition system which uses RFID tag array and neural networks to recognize gestures. By using an RFID tag array, we can obtain gesture information in a non-contact, non-infringing manner. By combining CNN and LSTM as CNN-LSTM, we can focus on both spatial and temporal features and get better performance. Experiments show that the accuracy of the system on the test set is 92.17%, and it performs well in recognizing different gestures of different users at different speeds.

Logistics monitoring is a fundamental application that utilizes RFID systems to manage numerous tagged-objects. Due to the frequent rearrangement of tagged-objects, a fast RFID-based tracking approach is highly desired for accurate logistics distribution. However, traditional RFID systems usually take tens of seconds to interrogate hundreds of RFID tags, not to mention the time delay involved to locate all the tags, which severely prevents from in-time tracking. To address this issue, we reduce the problem domain by first distinguishing the motion status of the tagged-objects, i.e., “stationary” or “moving”, and then tracking the moving objects with the state-of-the-art localization schemes, which significantly reduces the efforts of tracking all the objects. Toward this end, we propose a moving tag detection mechanism, which achieves the time efficiency by exploiting the useless collision signal in RFID systems. In particular, we extract two kinds of physical-layer features (namely, phase profile and backscatter link frequency) from the collision signal received by the USRP to distinguish tags at different positions. We further develop the Graph Matching (GM) method and Coherent Phase Variance (CPV) method to detect the moving tagged-objects. Experiment results show that our approach can accurately detect the moving objects while reducing 80 percent inventory time compared with the state-of-art solutions.

Embedding Radio-Frequency IDentification (RFID) into everyday objects to construct ubiquitous networks has been a long-standing goal. However, a major problem that hinders the attainment of this goal is the current inefficient reading of RFID tags. To address the issue, the research community introduces the technique of Bloom Filter (BF) to RFID systems. This work presents TagMap, a practical solution that acquires BFs across commercial off-the-shelf (COTS) RFID tags in the physical layer, enabling upper applications to boost their performance by orders of magnitude. The key idea is to treat all tags as if they were a single virtual sender, which hashes each tag into different intercepted inventories. Our approach does not require hardware nor firmware changes in commodity RFID tags - allows for rapid, zero-cost deployment in existing RFID tags. We design and implement TagMap reader with commodity device (e.g., USRP N210) platforms. Our comprehensive evaluation reveals that the overhead of TagMap is 66.22% lower than the state-of-the-art solution, with a bit error rate of 0.4%.

In conventional device-to-device (D2D) communication through wireless channels, it is an essential demand to authenticate with each other and establish spontaneous secure connections among the smart devices. In this paper, we propose an imitation-resistant mutual authentication and key generation framework for smart devices, by shaking these devices together. According to the multi-sensor data collected from smart devices, these devices are able to authenticate each other and generate a unique and consistent symmetric key if and only if they are shaken together. We have conducted comprehensive experimental study on shaking various devices, illustrated several novel observations and extracted some important clues for efficient key generation. We propose a series of novel techniques to make the key generation robust to noise and privacy-preserving, and generate highly distinctive and fully randomized symmetric keys among these devices. Realistic experiment results indicate that our solution is able to authenticate with each other and generate the symmetric keys with high accuracy and time-efficiency.

Nowadays, the growing demand for the 3D human-computer interaction (HCI) has brought about a number of novel approaches, which achieve the HCI by tracking the motion of different devices, including the translation and the rotation. In this paper, we propose to use a spinning linearly polarized antenna to track the 3D motion of a specified object attached with the passive RFID tag array. Different from the fixed antenna-based solutions, which suffer from the unavoidable signal interferences at some specific positions/orientations, and only achieve the good performance in some feasible sensing conditions, our spinning antenna-based solution seeks to sufficiently suppress the ambient signal interferences and extracts the most distinctive features, by actively spinning the antenna to create the optimal sensing condition. Moreover, by leveraging the matching/mismatching property of the linearly polarized antenna, i.e., in comparison to the circularly polarized antenna, the phase variation around the matching direction is more stable, and the RSSI variation in the mismatching direction is more distinctive, we are able to find more distinctive features to estimate the position and the orientation. We build a model to investigate the RSSI and the phase variation of the RFID tag along with the spinning of the antenna, and further extend the model from a single RFID tag to an RFID tag array. Furthermore, we design corresponding solutions to extract the distinctive RSSI and phase values from the RF-signal variation. Our solution tracks the translation of the tag array based on the phase features, and the rotation of the tag array based on the RSSI variation. The experimental results show that our system can achieve an average error of 13. 6cm in the translation tracking, and an average error of 8.3° in the rotation tracking in the 3D space.

Conventional spinning inspection systems, equipped with separated sensors (e.g., accelerometer, laser, etc.) and communication modules, are either very expensive and/or suffering from occlusion and narrow field of view. The recently proposed RFID-based sensing solution draws much attention due to its intriguing features, such as being cost-effective, applicable to occluded objects, auto-identification, etc. However, this solution only works in quiet settings where both the reader and spinning object remain absolutely stationary, as their shaking would ruin the periodicity and sparsity of the spinning signal, making it impossible to be recovered. To overcome such limitation, this work introduces Tagtwins, a robust spinning sensing system that can work in noisy settings. It addresses the challenge by attaching dual RFID tags on the spinning surface and developing a new formulation of spinning signal that is shaking-resilient, even if the shaking involves unknown trajectories. Our main contribution lies in two newly developed techniques. First, we propose relative spinning signal using dual tags' readings and analytically demonstrate its feasibility in various settings. Second, we introduce dual compressive reading to inspect high-frequency spinning with relatively low reading rate of RFIDs. We have implemented Tagtwins with commercial RFID devices and evaluated it extensively. Experimental results show that Tagtwins can inspect the rotation frequency with high accuracy and robustness.

With computer vision-based technologies, current Augmented reality (AR) systems can effectively recognize multiple objects with different visual characteristics. However, only limited degrees of distinctions can be offered among different objects with similar natural features, and inherent information about these objects cannot be effectively extracted. In this paper, we propose TaggedAR, i.e., an RFID-based approach to assist the recognition of multiple tagged objects in AR systems, by deploying additional RFID antennas to the COTS depth camera. By sufficiently exploring the correlations between the depth of field and the received RF-signal, we propose a rotate scanning-based scheme to distinguish multiple tagged objects in the stationary situation, and propose a continuous scanning-based scheme to distinguish multiple tagged human subjects in the mobile situation. By pairing the tags with the objects according to the correlations between the depth of field and RF-signals, we can accurately identify and distinguish multiple tagged objects to realize the vision of “tell me what I see” from the AR system. We have implemented a prototype system to evaluate the actual performance with case studies in a real-world environment. The experiment results show that our solution achieves an average match ratio of 91 percent in distinguishing up to dozens of tagged objects with a high deployment density.

Recognizing in-air hand gestures will benefit a wide range of applications such as sign-language recognition, remote control with hand gestures, and “writing” in the air as a new way of text input. This article presents AirContour, which focuses on in-air writing gesture recognition with a wrist-worn device. We propose a novel contour-based gesture model that converts human gestures to contours in 3D space and then recognizes the contours as characters. Different from 2D contours, the 3D contours may have the problems such as contour distortion caused by different viewing angles, contour difference caused by different writing directions, and the contour distribution across different planes. To address the above problem, we introduce Principal Component Analysis (PCA) to detect the principal/writing plane in 3D space, and then tune the projected 2D contour in the principal plane through reversing, rotating, and normalizing operations, to make the 2D contour in right orientation and normalized size under a uniform view. After that, we propose both an online approach, AC-Vec, and an offline approach, AC-CNN, for character recognition. The experimental results show that AC-Vec achieves an accuracy of 91.6% and AC-CNN achieves an accuracy of 94.3% for gesture/character recognition, both outperforming the existing approaches.

Nowadays, the demand for novel approaches of 2D human-computer interaction has enabled the emergence of a number of intelligent devices, such as Microsoft Surface Dial. Surface Dial realizes 2D interactions with the computer via simple clicks and rotations. In this paper, we propose RF-Dial, a battery-free solution for 2D human-computer interaction based on RFID tag arrays. We attach an array of RFID tags on the surface of an object, and continuously track the translation and rotation of the tagged object with an orthogonally deployed RFID antenna pair. In this way, we are able to transform an ordinary object like a board eraser into an intelligent HCI device. According to the RF-signals from the tag array, we build a geometric model to depict the relationship between the phase variations of the tag array and the rigid transformation of the tagged object, including the translation and rotation. By referring to the fixed topology of the tag array, we are able to accurately extract the translation and rotation of the tagged object during the moving process. Moreover, considering the variation of phase contours of the RF-signals at different positions, we divide the overall scanning area into the linear region and non-linear region in regard to the relationship between the phase variation and the tag movement, and propose tracking solutions for the two regions, respectively. We implemented a prototype system and evaluated the performance of RF-Dial in the real environment. The experiments show that RF-Dial achieved an average accuracy of 0. 6cm in the translation tracking, and an average accuracy of 1.9°in the rotation tracking.

Conventional spinning inspection systems, equipped with separated sensors (e.g., accelerometer, laser, etc.) and communication modules, are either very expensive and/or suffering from occlusion and narrow field of view. The recently proposed RFID-based sensing solution draws much attention due to its intriguing features, such as being cost-effective, applicable to occluded objects and auto-identification, etc. However, this solution only works in quiet settings where the reader and spinning object remain absolutely stationary, as their shaking would ruin the periodicity and sparsity of the spinning signal, making it impossible to be recovered. This work introduces Tagtwins, a robust spinning sensing system that can work in noisy settings. It addresses the challenge by attaching dual RFID tags on the spinning surface and developing a new formulation of spinning signal that is shaking-resilient, even if the shaking involves unknown trajectories. Our main contribution lies in two newly developed techniques, relative spinning signal and dual compressive reading. We analytically demonstrate that our solution can work in various settings. We have implemented Tagtwins with COTS RFID devices and evaluated it extensively. Experimental results show that Tagtwins can inspect the rotation frequency with high accuracy and robustness.

Recently, gesture recognition has gained considerable attention in emerging applications (e.g., AR/VR systems) to provide a better user experience for human-computer interaction. Existing solutions usually recognize the gestures based on wearable sensors or specialized signals (e.g., WiFi, acoustic and visible light), but they are either incurring high energy consumption or susceptible to the ambient environment, which prevents them from efficiently sensing the fine-grained finger movements. In this paper, we present RF-finger, a device-free system based on Commercial-Off-The-Shelf (COTS) RFID, which leverages a tag array on a letter-size paper to sense the fine-grained finger movements performed in front of the paper. Particularly, we focus on two kinds of sensing modes: finger tracking recovers the moving trace of finger writings; multi-touch gesture recognition identifies the multi-touch gestures involving multiple fingers. Specifically, we build a theoretical model to extract the fine-grained reflection feature from the raw RF -signal, which describes the finger influence on the tag array in cm- level resolution. For the finger tracking, we leverage K-Nearest Neighbors (KNN) to pinpoint the finger position relying on the fine-grained reflection features, and obtain a smoothed trace via Kalman filter. Additionally, we construct the reflection image of each multi-touch gesture from the reflection features by regarding the multiple fingers as a whole. Finally, we use a Convolutional Neural Network (CNN) to identify the multi-touch gestures based on the images. Extensive experiments validate that RF -finger can achieve as high as 88% and 92% accuracy for finger tracking and multi-touch gesture recognition, respectively.

Nowadays, novel approaches of 3D human-computer interaction have enabled the capability of manipulating in the 3D space rather than 2D space. For example, Microsoft Surface Pen leverages the embedded sensors to sense the 3D manipulations, such as inclining the pen to get bolder handwriting. In this paper, we propose RF-Brush, a battery-free and light-weight solution for 3D human-computer interaction based on RFID, by simply attaching a linear RFID tag array onto the linear shaped object like a brush. RF-Brush senses the 3D orientation and 2D movement of the linear shaped object, when the human subject is drawing with this object in the 3D space. Here, the 3D orientation refers to the relative orientation of the linear shaped object to the operating plane, whereas the 2D movement refers to the moving trace in the 2D operating plane. In this way, we are able to transform an ordinary linear shaped object like a brush or pen to an intelligent HCI device. Particularly, we build two geometric models to depict the relationship between the RF-signal and the 3D orientation as well as 2D movement, respectively. Based on the geometric model, we propose the linear tag array-based HCI solution, implemented a prototype system, and evaluated the performance in real environment. The experiments show that RF-Brush achieves an average error of 5.7° and 8.6° of elevation and azimuthal angle, respectively, and an average error of 3.8cm and 4.2cm in movement tracking along X-axis and Y-axis, respectively. Moreover, RF-Brush achieves 89% in letter recognition accuracy.

Enabling touch gesture sensing on all surfaces of the mobile device, not limited to the touchscreen area, leads to new user interaction experiences. In this paper, we propose VSkin, a system that supports fine-grained gesture-sensing on the back of mobile devices based on acoustic signals. VSkin utilizes both the structure-borne sounds, i.e., sounds propagating through the structure of the device, and the air-borne sounds, i.e., sounds propagating through the air, to sense finger tapping and movements. By measuring both the amplitude and the phase of each path of sound signals, VSkin detects tapping events with an accuracy of 99.65% and captures finger movements with an accuracy of 3.59mm.

Infusion monitoring is in great demand for the hospital. In this demo, we propose RF-iCare, an RFID-based approach for monitoring the infusion status, including the liquid level and the drop speed. With a tag array attached on the infusion bottle, we design an RSSI-based signal match model to estimate the liquid level. With a tag attached on the Murphy's dropper, we leverage the phase variation of the tag to estimate the drop speed. We implement RF-iCare with a COTS RFID system and evaluate it in the real-world hospitals. Our experiments demonstrate that RF-iCare can accurately monitor the completion of the infusion over 91% tests, and estimate the liquid level with the mean accuracy of 0.8 cm as well as the drop speed with the error rate less than 3%.

Because of the smaller size of mobile devices, text entry with on-screen keyboards becomes inefficient. Therefore, we present CamK, a camera-based text-entry method, which can use a panel (e.g., a piece of paper) with a keyboard layout to input text into small devices. With the built-in camera of the mobile device, CamK captures images during the typing process and utilizes image processing techniques to recognize the typing behavior, i.e., extract the keys, track the user's fingertips, detect, and locate keystrokes. To achieve high accuracy of keystroke localization and low false positive rate of keystroke detection, CamK introduces the initial training and online calibration. To reduce the time latency, CamK optimizes computation-intensive modules by changing image sizes, focusing on target areas, introducing multiple threads, removing the operations of writing or reading images. Finally, we implement CamK on mobile devices running Android. Our experimental results show that CamK can achieve above 95 percent accuracy in keystroke localization, with only a 4.8 percent false positive rate. When compared with on-screen keyboards, CamK can achieve a 1.25X typing speedup for regular text input and 2.5X for random character input. In addition, we introduce word prediction to further improve the input speed for regular text by 13.4 percent.

The human-computer interactions have moved from the conventional approaches of entering inputs into the keyboards/touchpads to the brand-new approaches of performing interactions in the air. In this paper, we propose RF-glove, a system that recognizes concurrent multiple finger micromovement using RF signals, so as to realize the vision of “multi-touch in the air.” It uses a commercial-off-the-shelf (COTS) RFID reader with three antennas and five COTS tags attached to the five fingers of a glove, one tag per finger. During the process of a user performing finger micromovements, we let the RFID reader continuously interrogate these tags and obtain the backscattered RF signals from each tag. For each antenna-tag pair, the reader obtains a sequence of RF phase values called a phase profile from the tag's responses over time. To tradeoff between accuracy and robustness in terms of matching resolution, we propose a two phase approach, including coarse-grained filtering and fine-grained matching. To tackle the variation of template phase profiles at different positions, we propose a phase-model-based solution to reconstruct the template phase profiles based on the exact locations. Experiment results show that we achieve an average accuracy of 92.1% under various moving speeds, orientation deviations, and so on.

While the mobile users enjoy the anytime anywhere Internet access by connecting their mobile devices through Wi-Fi services, the increasing deployment of access points (APs) have raised a number of privacy concerns. This paper explores the potential of smartphone privacy leakage caused by surrounding APs. In particular, we study to what extent the users' personal information such as social relationships and demographics could be revealed leveraging simple signal information from APs without examining the Wi-Fi traffic. Our approach utilizes users' activities at daily visited places derived from the surrounding APs to infer users' social interactions and individual behaviors. Furthermore, we develop two new mechanisms: the Closeness-based Social Relationships Inference algorithm captures how closely people interact with each other by evaluating their physical closeness and derives fine-grained social relationships, whereas the Behavior-based Demographics Inference method differentiates various individual behaviors via the extracted activity features (e.g., activeness and time slots) at each daily place to reveal users' demographics. Extensive experiments conducted with 21 participants' real daily life including 257 different places in three cities over a 6-month period demonstrate that the simple signal information from surrounding APs have a high potential to reveal people's social relationships and infer demographics with an over 90% accuracy when using our approach.

Activity sensing has become a key technology for many ubiquitous applications, such as exercise monitoring and elder care. Most traditional approaches track the human motions and perform activity recognition based on the waveform matching schemes in the raw data representation level. In regard to the complex activities with relatively large moving range, they usually fail to accurately recognize these activities, due to the inherent variations in human activities. In this paper, we propose a wearable approach for logic cognition-based activity sensing scheme in the logical representation level, by leveraging the meta-activity recognition. Our solution extracts the angle profiles from the raw inertial measurements, to depict the angle variation of limb movement in regard to the consistent body coordinate system. It further extracts the meta-activity profiles to depict the sequence of small-range activity units in the complex activity. By leveraging the least edit distance-based matching scheme, our solution is able to accurately perform the activity sensing. Based on the logic cognition-based activity sensing, our solution achieves lightweight-training recognition, which requires a small quantity of training samples to build the templates, and user-independent recognition, which requires no training from the specific user. The experiment results in real settings shows that our meta-activity recognition achieves an average accuracy of 92% for user-independent activity sensing.

In this paper, we propose 3DLoc, which performs 3-dimensional localization on the tagged objects by using the RFID tag arrays. 3DLoc deploys three arrays of RFID tags on three mutually orthogonal surfaces of each object. When performing 3D localization, 3DLoc continuously moves the RFID antenna and scans the tagged objects in a 2-dimensional space right in front of the tagged objects. It then estimates the object's 3D position according to the phases from the tag arrays. By referring to the fixed layout of the tag array, we use Angle of Arrival-based schemes to accurately estimate the tagged objects' orientation and 3D coordinates in the 3D space. To suppress the localization errors caused by the multipath effect, we use the linear relationship of the AoA parameters to remove the unexpected outliers from the estimated results. We have implemented a prototype system and evaluated the actual performance in the real complex environment. The experimental results show that 3DLoc achieves the mean accuracy of 10cm in free space and 15.3cm in the multipath environment for the tagged object.

Nowadays, 3D reconstruction has been introduced in monitoring the package placement in logistic industry-related applications. Existing 3D econstruction methods are mainly based on computer vision or sensor-based approaches, which are limited by the line-of-sight or battery life constraint. In this paper, we propose RF-3DScan to perform 3D reconstruction on tagged packages via passive RFID, by attaching multiple reference tags onto the surface of the packages. The basic idea is that by moving the antenna along straight lines within a constrained 2-dimensional space, the antenna obtains the RF-signals of the reference tags attached on the packages. By extracting the phase differences to build the angle profile for each tag, RF-3DScan can compare the angle profiles of the different reference tags and derive their relative positions, then further determine the package orientation and stacking for 3D reconstruction. We implement RF- 3DScan and evaluate its performance in real settings. The experiment results show that the average identification accuracy of the bottom face is about 92.5%, and the average estimation error of the rotation angle is about 4.08o.

Nowadays, Unmanned Aerial Vehicles(UAVs) have been widely applied in our life. However, the existing approach of interacting with UAVs, i.e., using a remote controller with control sticks, is not a natural and intuitive way. In this paper, we present a novel approach for users to interact with personal UAVs using wearable devices. The basic idea of our approach is to manipulate UAVs based on human activity sensing, including motion recognition and pedestrian dead- reckoning. We have implemented the proposed approach on a DJI drone, and evaluated its performance in the real- world environment. Realistic experiment results show that our solution can replace the remote controller to manipulate the UAV.

Due to the portability of smart phones, more and more people tend to take photos with smart phones. However, energy-saving continues to be a thorny problem, since photographing is a rather power hungry function. To extend the battery life of phones while taking photos, we propose a context-aware energy-saving scheme called “SenSave.” SenSave senses the user’s activities during photographing and adopts suitable energy-saving strategies accordingly. SenSave works based on the observation that a lot of energy during photographing is wasted in preparations before shooting. By leveraging the low power-consuming embedded sensors, such as accelerometer and gyroscope, we can recognize the user’s activities and reduce unnecessary energy consumption. Besides, by maintaining an activity state machine, SenSave can determine the user’s activity progressively and improve the recognition accuracy. Experiment results show that SenSave can recognize the user’s activities with an average accuracy of 95.5% and reduce the energy consumption during photographing by 30.0%, when compared to the approach by frequently turning ON/OFF the camera or screen. Additionally, we enhance “SenSave” by introducing an extended Markov chain to predict the next activity state and adopt the energy-saving strategy in advance. Then, we can reduce the energy consumption during photographing by 36.1%.

Nowadays, people usually depend on augmented reality (AR) systems to obtain an augmented view in a real-world environment. With the help of advanced AR technology (e.g. object recognition), users can effectively distinguish multiple objects of different types. However, these techniques can only offer limited degrees of distinctions among different objects and cannot provide more inherent information about these objects. In this paper, we leverage RFID technology to further label different objects with RFID tags. We deploy additional RFID antennas to the COTS depth camera and propose a continuous scanning-based scheme to scan the objects, i.e., the system continuously rotates and samples the depth of field and RF-signals from these tagged objects. In this way, by pairing the tags with the objects according to the correlations between the depth of field and RF-signals, we can accurately identify and distinguish multiple tagged objects to realize the vision of "tell me what I see" from the augmented reality system. For example, in front of multiple unknown people wearing RFID tagged badges in public events, our system can identify these people and further show their inherent information from the RFID tags, such as their names, jobs, titles, etc. We have implemented a prototype system to evaluate the actual performance. The experiment results show that our solution achieves an average match ratio of 91% in distinguishing up to dozens of tagged objects with a high deployment density.

In this paper, we leverage RFID technology to label different objects with RFID tags, so as to realize the vision of "show me what I see from the augmented reality system". We deploy additional RFID antennas to the COTS depth camera and propose a continuous scanning-based scheme to scan the objects, i.e., the system continuously rotates and samples the depth of field and RF-signals from these tagged objects. In this way, we can accurately identify and distinguish multiple tagged objects, by pairing the tags with the objects according to the correlations between the depth of field and RF-signals. Our solution achieves an average match ratio of 91% in distinguishing up to dozens of tagged objects with a high deployment density.

In a number of RFID-based applications such as logistics monitoring, the RFID systems are deployed to monitor a large number of RFID tags. They are usually required to track the movement of all tags in a real-time approach, since the tagged-goods are moved in and out in a rather frequent approach. However, a typical cycle of tag inventory in COTS RFID system usually takes tens of seconds to interrogate hundreds of RFID tags. This hinders the system to track the movement of all tags in time. One critical issue in such type of tag monitoring is to efficiently distinguish the motion status of all tags, i.e., stationary or moving. According to the motion status of different tags, the state-of-art localization schemes can further track those moving tags, instead of tracking all tags. In this paper, we propose a real-time approach to detect the moving tags in the monitoring area, which is a fundamental premise to support tracking the movement of all tags. We achieve the time efficiency by decoding collisions from the physical layer. Instead of using the EPC ID, which cannot be decoded in collision slots, we are able to extract two kinds of physical-layer features of RFID tags, i.e., the phase profile and the backscatter link frequency, to distinguish among different tags in different positions. By resolving the two physical-layer features from the tag collisions, we are able to derive the motion status of multiple tags simultaneously, and greatly improve the time-efficiency. Experiment result shows that our solution can accurately detect the moving tags while reducing 80% of inventory time compared with the state-of-art solutions.

Due to the smaller size of mobile devices, on-screen keyboards become inefficient for text entry. In this paper, we present CamK, a camera-based text-entry method, which uses an arbitrary panel (e.g., a piece of paper) with a keyboard layout to input text into small devices. CamK captures the images during the typing process and uses the image processing technique to recognize the typing behavior. The principle of CamK is to extract the keys, track the user's fingertips, detect and localize the keystroke. To achieve high accuracy of keystroke localization and low false positive rate of keystroke detection, CamK introduces the initial training and online calibration. Additionally, CamK optimizes computation-intensive modules to reduce the time latency. We implement CamK on a mobile device running Android. Our experiment results show that CamK can achieve above 95% accuracy of keystroke localization, with only 4.8% false positive keystrokes. When compared to on-screen keyboards, CamK can achieve 1.25X typing speedup for regular text input and 2.5X for random character input.

Currently, conventional indoor localization schemes mainly leverage WiFi-based or Bluetooth-based schemes to locate the users in the indoor environment. These schemes require to deploy the infrastructures such as the WiFi APs and Bluetooth beacons in advance to assist indoor localization. This property hinders the indoor localization schemes in that they are not scalable to any other situations without these infrastructures. In this paper, we propose FootStep-Tracker, an anchor-free indoor localization scheme purely based on sensing the user's footsteps. By embedding the tiny SensorTag into the user's shoes, FootStep-Tracker is able to accurately perceive the user's moving trace, including the moving direction and distance, by leveraging the accelerometers and gyroscopes. Furthermore, by detecting the user's activities such as ascending/descending the stairs and taking an elevator, FootStep-Tracker can effectively correlate with the specified positions such as stairs and elevators, and further determine the exacted moving traces in the indoor map by leveraging the space constraints in the map. Realistic experiment results show that, FootStep-Tracker is able to achieve an average localization accuracy of 1m for indoor localization, without any infrastructures having been deployed in advance.

With the rapid proliferation of RFID technologies, RFID has been introduced into applications such as inventory and sampling inspection. Conventionally, in RFID systems, the reader usually identifies all the RFID tags in the interrogation region with the maximum power. However, some applications may only need to identify the tags in a specified area, which is usually smaller than the reader's default interrogation region. An example could be identifying the tags in a box, while ignoring the tags out of the box. In this paper, we respectively present two solutions to identify the tags in the specified area. The principle of the solutions can be compared to the picture-taking process of an auto-focus camera, which firstly focuses on the target automatically and then takes the picture. Similarly, our solutions first focus on the specified area and then shoot the tags. The design of the two solutions is based on the extensive empirical study on RFID tags. Realistic experiment results show that our solutions can reduce the execution time by 44 percent compared to the baseline solution, which identifies the tags with maximum power. Furthermore, we improve the proposed solutions to make them work well in more complex environments.

Heterogeneous cellular networks use small base stations, such as femtocells and WiFi APs, to offload traffic from macrocells. While network operators wish to globally balance the traffic, users may selfishly select the nearest base stations and make some base stations overcrowded. In this paper, we propose to use an auction-based algorithm - Femto-Matching, to achieve both load balancing among base stations and fairness among users. Femto-Matching optimally solves the global proportional fairness problem in polynomial time by transforming it into an equivalent matching problem. Furthermore, it can efficiently utilize the capacity of randomly deployed small cells. Our trace-driven simulations show Femto-Matching can reduce the load of macrocells by more than 30% compared to non-cooperative game based strategies.

Nowadays more and more users tend to take photos with their smart phones. However, energy-saving continues to be a thorny problem for smart camera phones, since smart phone photographing is a very power hungry function. In this paper, we propose a context aware energy-saving scheme for smart camera phones, by accurately sensing the user's activities in the photographing process. Our solution is based on the observation that during the process of photographing, most of the energy are wasted in the preparations before the shooting. By leveraging the embedded sensors like the accelerometer and gyroscope, our solution is able to extract representative features to perceive the user's current activities including body movement, arm movement and wrist movement. Furthermore, by maintaining an activity state machine, our solution can accurately determine the user's current activity states and make the corresponding energy saving strategies. Experiment results show that, our solution is able to perceive the user's activities with an average accuracy of 95.5% and reduce the overall energy consumption by 46.5% for smart camera phones compared to that without energy-saving scheme.

In this paper, we show the first comprehensive experimental study on mobile RFID reading performance based on a relatively large number of tags. By making a number of observations regarding the tag reading performance, we build a model to depict how various parameters affect the reading performance. Through our model, we have designed very efficient algorithms to maximize the time-efficiency and energy-efficiency by adjusting the reader's power and moving speed. Our experiments show that our algorithms can reduce the total scanning time by 50 percent and the total energy consumption by 83 percent compared to the prior solutions.

Collecting histograms over RFID tags is an essential premise for effective aggregate queries and analysis in large-scale RFID-based applications. In this paper we consider an efficient collection of histograms from the massive number of RFID tags, without the need to read all tag data. In order to achieve time efficiency, we propose a novel, ensemble sampling-based method to simultaneously estimate the tag size for a number of categories. We first consider the problem of basic histogram collection, and propose an efficient algorithm based on the idea of ensemble sampling. We further consider the problems of advanced histogram collection, respectively, with an iceberg query and a top-k query. Efficient algorithms are proposed to tackle the above problems such that the qualified/unqualified categories can be quickly identified. This ensemble sampling-based framework is very flexible and compatible to current tag-counting estimators, which can be efficiently leveraged to estimate the tag size for each category. Experiment results indicate that our ensemble sampling-based solutions can achieve a much better performance than the basic estimation/identification schemes.

With the rapid proliferation of RFID-based applications, RFID tags have been deployed into pervasive spaces in increasingly large numbers, e.g., the shelves of super markets are filled with tag-labeled items. Conventional localization schemes usually leverage precise anchor nodes to help compute the position of objects. However, it is usually difficult to find or deploy enough anchor nodes for accurate localization. In this paper, we propose solutions to locate the mobile users based on imprecise anchors in RFID systems. A large number of tags with approximate locations are used as anchor nodes to compute the user's locations. We thus present a time-efficient localization scheme to continuously tracking the mobile users. Experimental results indicate that our solutions can accurately locate the mobile users in a real-time approach. The improved method's accuracy is more than 30% better than the base solution.

Collecting histograms over RFID tags is an essential premise for effective aggregate queries and analysis in large-scale RFID-based applications. In this paper we consider efficient collection of histograms from the massive number of RFID tags without the need to read all tag data. We first consider the problem of basic histogram collection and propose an efficient algorithm based on the idea of ensemble sampling. We further consider the problems of advanced histogram collection, respectively, with an iceberg query and a top-k query. Efficient algorithms are proposed to tackle the above problems such that the qualified/unqualified categories can be quickly identified. Experiment results indicate that our ensemble sampling-based solutions can achieve a much better performance than the basic estimation/identification schemes.

With the rapid proliferation of Wi-Fi technologies in recent years, it has become possible to utilize the vehicular wireless network to assist the route guidance for drivers in a cooperative approach, aiming to mitigating heavy traffic congestion. In this paper, we investigate into the route guidance problem in vehicular wireless network, and then propose two efficient routing algorithms, i.e., centralized route guidance and distributed route guidance, according to different situations. A hybrid framework is then proposed to provide optimized routing decisions in a uniform way. Simulation results in Simulation of Urban MObility (SUMO) indicate that, our route guidance schemes achieve much better performance than traditional GPS-based navigation and randomized routing.

In real life, looking for a misplaced object like a key in the room can be usually like searching for a needle in a haystack. In this paper, we propose a novel solution to accurately locate the specified objects attached with RFID tags in indoor environments, by efficiently leveraging the RFID technology. By making a number of novel observations regarding the tag reading performance, we obtain several regularities to depict how various parameters including the reader's power and the antenna's scanning angle affect the reading performance. Based on the regularities, we have designed very efficient algorithms to maximize the accuracy and the time-efficiency for localization. Without the help of any anchor nodes, our solution can rapidly navigate to the target object from a specific initial position. We have implemented a system prototype to evaluate the actual performance in realistic applications. The realistic experiment results show that our solution can restrict the average localization error within 49 cm and reduce the total navigation time by 33% compared to the baseline solutions.

ABSTRACT This paper addresses radio frequency identification (RFID) authentication and ownership transfer in offline scenarios. Four typical related works are reviewed in detail. A series of shortcomings and vulnerabilities of them are pointed out. A new RFID authentication protocol based on a novel tag‐owner‐assisting architecture is proposed, making a tag's owner an essential participant of the RFID authentication process. The proposed protocol is distinguished from existing works in providing ownership transfer, access control, and mutual authentication without any centralized database neither on a backend server nor in a reader. The security of the proposed protocol is verified by using automated validation of Internet security protocols and applications tool. The proposed protocol is server‐less, simple, scalable, untraceable, and device‐independent. These features are simultaneously achieved in a single RFID authentication protocol for the first time. Copyright © 2014 John Wiley & Sons, Ltd.

As a supporting technology for most pervasive applications, indoor localization and navigation has attracted extensive attention in recent years. Conventional solutions mainly leverage techniques like WiFi, cellular network etc. to effectively locate the user for indoor localization and navigation. In this paper, we investigate into the problem of indoor navigation by using the RFID-based delay tolerant network. Being different from the previous work, we aim to efficiently locate and navigate to a specified mobile user who is continuously moving within the indoor environment. We respectively propose a framework to schedule the tasks and manage the resources in the network and a navigation algorithm to locate and navigate to the moving target. Experiment results show that our solution can efficiently reduce the average searching time for indoor navigation.

With the rapid proliferation of RFID technologies, RFID has been introduced to the applications like safety inspection and warehouse management. Conventionally a number of deployment rules are specified for these applications. This paper studies a practically important problem of rule checking over a large set of RFID tags, i.e., checking whether the specified rules are satisfied according to the RFID tags within the monitoring area. This rule checking function may need to be executed frequently over a large number of tags and therefore should be made efficient in terms of execution time. Aiming to achieve time efficiency, we propose two efficient protocols based on the collision detection and the logical features of rules, respectively. Simulation results indicate that our protocols achieve much better performance than other solutions in terms of time efficiency.

In many pervasive applications like the intelligent bookshelves in libraries, it is essential to accurately locate the items to provide the location-based service, e.g., the average localization error should be smaller than 50 cm and the localization delay should be within several seconds. Conventional indoor-localization schemes cannot provide such accurate localization results. In this paper, we design an adaptive, accurate indoor-localization scheme using passive RFID systems. We propose two adaptive solutions, i.e., the adaptive power stepping and the adaptive calibration, which can adaptively adjust the critical parameters and leverage the feedbacks to improve the localization accuracy. The realistic experiment results indicate that, our adaptive localization scheme can achieve an accuracy of 31 cm within 2.6 seconds on average.

In this paper, we show the first comprehensive experimental study on mobile RFID reading performance based on a relatively large number of tags. By making a number of observations regarding the tag reading performance, we build a model to depict how various parameters affect the reading performance. Through our model, we have designed very efficient algorithms to maximize the time-efficiency and energy-efficiency by adjusting the reader's power and moving speed. Our experiments show that our algorithms can reduce the total scanning time by 50\% and the total energy consumption by 83\% compared to the prior solutions.

It is a tedious task to search and locate a specific book from massive number of books arbitrarily placed in a bookshelf. In this paper, we demonstrate iBookshelf, a system which allows users to quickly search and accurately locate books in the bookshelf, by leveraging a passive RFID system. By deploying a number of reference tags on the bookshelf, we are able to perform localization based on the similarities in received signal strength, and effectively offset the impact from the ambient noises and interferences. We deploy and evaluate our system in a real 3m x 2.5m bookshelf, and show that users are able to locate the book from our Android-based application with 85% accuracy.

Deploying road-side WiFi access points has made possible internet access in a vehicle, nevertheless it is challenging to maintain client performance at vehicular speed especially when multiple mobile users exist. This paper considers the association control problem for vehicular WiFi access in the Drive-thru Internet scenario. In particular, we aim to improve the efficiency and fairness for all users. We design efficient algorithms to achieve these objectives through several techniques including approximation. Our simulation results demonstrate that our algorithms can achieve significantly better performance than conventional approaches.

In this paper we consider how to efficiently identify tags on the moving conveyor. Considering conditions like the path loss and multi-path effect in realistic settings, we first propose a probabilistic model for RFID tag identification. Based on this model, we propose efficient solutions to identify moving RFID tags, according to the fixed-path mobility on the conveyor. A dynamic program based solution and an adaptive solution are proposed to select optimized frame sizes during the query cycles. Simulation results indicate that by leveraging the probabilistic model our solutions can achieve much better performance than using parameters for the ideal propagation situations.

Deploying city-wide 802.11 access points has made possible internet access in a vehicle, nevertheless it is challenging to maintain client performance at vehicular speed especially when multiple mobile users exist. This paper considers the association control problem for vehicular networks in drive-thru Internet scenarios. In particular, we aim to improve the overall throughput and fairness for all users. We design efficient algorithms to achieve the objectives through several techniques including approximation. Our simulation results confirm the performance of our algorithms.

This paper presents the design of a decentralized storage scheme to support multi-dimensional range queries over sensor networks. We build a distributed k-d tree based index structure over sensor network, so as to efficiently map high dimensional event data to a two-dimensional space of sensors while preserving the proximity of events. We propose a dynamic programming based methodology to control the granularity of the index tree in an optimized approach, and an optimized routing scheme for range query processing to achieve best energy efficiency. The simulation results demonstrate the efficiency of the design.

The fundamental characteristics of SOA, loose coupling and on-demand integration, enable organizations to seek more flexibility and responsiveness from their business IT systems. However, this brings challenges to assure QoS, especially availability, which should be considered in an integrated way in an SOA environment. Traditionally, availability is measured for each IT resource, but within SOA environments, rather than being considered individually, availability should be analyzed from an end-to-end view from both business and IT perspectives. In this paper, to address the availability problem of SOA, we propose a methodology that analyzes availability weak points in SOA deployment frameworks, leveraging workflow definitions that specify availability requirements at business level. This methodology includes an effective way to calculate high availability enhancement recommendations for a given SOA deployment topology with near-minimum cost, while meeting the business-level availability requirements. A prototype has been implemented as an extension to IBM's SOA deployment framework. Its efficiency and performance are analyzed here.

Data-centric storage is an effective and important technique in sensor networks, however, most data-centric storage schemes may not be energy efficient and load balanced due to non-uniform event and query distributions. This paper proposes EEBASS, it utilizes an approximation algorithm to solve the optimal storage placement problem according to the variance of event and query distributions, aiming to minimize the total energy consumption for data-centric storage scheme. And it further leverages a ring based replication structure to achieve the load balance goal. Simulation results show that EEBASS is more energy efficient and balanced than traditional data-centric storage mechanisms in sensor network.

Availability is one of the important factors to be considered for business-driven IT service management. This paper addresses the issue of analyzing what we call availability weak-points in an SOA deployment framework, leveraging workflow definitions to specify the high availability requirement at the business process level. In our weak-point analysis framework, we present an effective analysis methodology to calculate the optimal high availability solution with minimum cost, while meeting the business level availability requirements. We evaluate the weakpoint analysis methodology, and show that our methodology can identify a near-optimal solution for availability enhancement over the SOA deployment framework.