Radio Frequency Identification (RFID) technology has emerged as a pervasive modality for human motion sensing in applications such as smart environments and healthcare monitoring. However, the inherent through-wall sensing capability of RFID technology raises critical privacy concerns regarding the unintended leakage of human motion information, a challenge that has not been adequately addressed. To fill this gap, we present a metasurface-based RFID sensing defence (MetaRFence), the first system designed to protect human motion privacy against adversarial through-wall RFID sensing. To this end, we first devise a programmable metasurface comprising 1-bit phase shifters to systematically obfuscate motion-induced signal patterns. Then, we characterize the metasurface's impact on RFID signals across temporal and spectral domains through comprehensive theoretical modeling and empirical investigations. However, our analysis reveals that it is non-trivial to achieve effective signal obfuscation in both domains, primarily due to a fundamental trade-off between increasing temporal signal variation and masking human motion in its spectrum. To overcome this, we judiciously devise a metasurface controlling strategy that jointly optimizes the signal entropy, variance, and spectrum distribution to reach a balance between temporal and spectral motion obfuscation. Our comprehensive experiments demonstrate thatMetaRFencereduces adversarial through-wall motion detection rates to$\leq$6%, decreases the F1-score of human gesture recognition to$\leq$0.11 on average, and amplifies respiration rate estimation errors by 3×, establishing a robust defense mechanism for RFID-based motion privacy protection.

Nowadays, video stream analysis has a wide range of applications, and videos captured by mobile devices are often used for such analyses. Stable videos provide a better foundation for downstream analysis. Current stabilization methods usually estimate camera motion using feature points, but moving objects can be mistaken for camera movement. Moreover, the loss of depth can cause image distortion during video smoothing. To address above challenges, we propose a multi-modal video stabilization method for dynamic scenes, assisted by the gyroscope embedded in the mobile device. Firstly, we notice that the rotation of feature points extracted from trees, buildings or other static objects, should be consistent with camera rotation, while feature points extracted from cars and other dynamic objects have completely different rotation. Hence, we cluster feature points and calculate the rotation for each cluster, then compare their rotation with gyroscope rotation to distinguish static feature points. Secondly, as it takes much time to calculate the absolute depth of feature points, we suggest leveraging gyro measurements to estimate the relative depth of feature points in a light-weight way. Furthermore, we can classify feature points by relative depth, allowing us to apply different smoothing techniques to points at different depths, thereby enhancing the video stabilization effect. Experimental results show that, compared to traditional multimodal methods, we can improve video stability (SSIM value) by 47.8% and increase processing speed by 37%, effectively enhancing the quality of shaky videos in dynamic and loss of depth scenes while achieving greater efficiency.

As the low-altitude economy expands, Unmanned Aerial Vehicles (UAVs) have become essential across a variety of applications. However, non-cooperative UAVs threaten personal privacy and public safety, especially when carrying dangerous payloads. Current UAV detection methods primarily rely on computer vision, which often fails in severe weather conditions. This paper introduces mmUAVsense, a mmWave-based system designed for detecting and identifying UAVs in complex environments. In addition to detecting UAVs, mmUAVsense can also estimate the rotational speed of their propellers, which offers valuable information for assessing the potential payload. A key challenge in detecting UAVs is their small Radar Cross Section (RCS), which makes them easily masked by the environment, especially when flying near large objects. To address this issue, we propose a Moving Target Indicator (MTI) to calculate the time-spatial difference, which helps suppress static environmental clutter and make the dynamic reflection from the UAV more noticeable. To differentiate UAVs from other flying objects, we exploit the unique harmonic feature in the UAV’s Doppler spectrum, caused by the rapid spinning of its propellers, to identify UAVs among other airborne objects. To estimate the UAV’s rotational speed, we analyze the relationship between harmonic features and propeller speed, and propose to use Chirp-Z Transformation (CZT) to accurately extract the frequency from the spectrum. We have implemented a system prototype in various outdoor environments, and extensive experiments demonstrate that our system can identify UAVs with over 95% accuracy, with an average rotational speed error of just 3.8%. Based on the existing UAV model, our rotary-sensing-based system can sense the UAV’s extra load weight with only 8 grams of average error.

Image stitching is often affected by obstructions such as pedestrians and vehicles. Traditional methods often ignore obstructions or retain them, leading to artifacts. They also cause edge distortion due to fixed-perspective stitching. To address above issues, this poster presents IIS, an IMU-assisted Image Stitching method. It uses IMU data for perspective pre-alignment, projecting images to an intermediate perspective to reduce edge distortion. It simultaneously removes obstructions and mitigating their impact. A weighted target scene stitching strategy is integrated to further enhance stitching quality. IIS effectively reduces artifacts and maintains high computational efficiency.

Unmanned aerial vehicle (UAV) poses a major threat to airspace security and privacy protection. However, existing UAV identification systems face the limitations of feature extraction quality degradation under low signal-to-noise ratio (SNR) conditions, and the average delay of traditional tracking algorithms is difficult to meet the real-time requirements due to high computational complexity. To address these issues, this paper proposes mmRTD, a real-time identification and tracking system for non-cooperative UAV sensing using mmWave radar. To solve the problem of serious feature extraction distortion in low SNR environment, this paper proposes a dynamic period adjustment mechanism driven by spectrum volatility to significantly improve the quality of feature extraction under low SNR conditions. To solve the problem of insufficient real-time performance of traditional tracking methods, this paper proposes a lightweight frame-based tracking framework through the target state detection method to solve the real-time bottleneck caused by high-dimensional signal processing of traditional methods. The experimental results show that the detection accuracy of mmRTD reaches 97.2% within a range of 40 m. In particular, the first detection time of mmRTD is two orders of magnitude higher than that of the traditional scheme, which verifies its potential for real-time identification in complex scenes.

Distributed multi-exit neural networks (MeNNs) enable mobile devices to handle complex tasks such as image classification, but their performance is highly dependent on transmission quality and is therefore vulnerable to side-channel attacks. In this paper, we design a side-channel attack model and propose an efficient inference framework based on the distributed MeNN to resist the designed attack. First, we design an intelligent side-channel attack model, in which the attacker can eavesdrop on the communication channel and use deep reinforcement learning (RL) to predict the early exit decision of each sample. Next, we develop a defense method that employs a hierarchical and multi-agent RL to determine whether to infer locally or offload to a chosen early exit on the server, and to adjust the transmit power accordingly. We further propose a critic-guided safety mechanism that steers local agents away from risky policies that would cause inference failures or severe data leakage. We prove that our framework enforces a strict instantaneous security constraint and asymptotically achieves the optimum by deriving a regret bound. Extensive experiments on several datasets (including CIFAR‑10, CIFAR‑100, STL‑10, EMNIST, FMNIST, and Stanford Cars) show that our method reduces inference latency, improves classification accuracy, and significantly enhances robustness against side-channel attacks, as compared with two benchmarks SCAN and PCE.

With the rise of intelligent systems like assisted driving and robotics, all-weather target identification and 3D localization systems have become crucial for reliable obstacle avoidance and navigation. However, vision-based methods struggle to provide accurate target locations under low light or bad weather. Radar-based solutions like mmWave radar and LiDAR are robust but hindered by high costs and challenges in recognizing target identities at scale. In this paper, we propose alow-cost, all-weather target identification and 3D localization systembased onLED-tags, which system can address the needs of intelligent systems for obstacle avoidance in complex environments. We explore the backscatter communication of LED devices and design adual-modal LED-Tag, which includes two features: a backscatter RF signal detectable by RF devices and visual light spot information detectable by cameras, both sharing the same ID. To enhance the limited backscatter capability, we propose amulti-branch parallel modelthat enhances the signal strength using beamforming synthesis and achannel adjustment mechanismto improve robustness in complex environments, ensuring accurate 3D localization. For multi-target identification, we design an LED-tag encoding system, assigning each tag a unique encoding sequence. Each target's identity can be recognized with our customizedID decoding method, which leverages prior information and time-domain sampling characteristics. Extensive experimental results show that the backscatter communication and target detection range of LED-tags can reach15m. Moreover, the system achieves anaverage localization error of 7.3cm within a 5m range, demonstrating the system's excellent performance in terms of practicality and accuracy.

Image-based indoor localization using smartphones has become popular, leveraging visual landmarks and fingerprint extraction for localization. Fingerprint density significantly affects accuracy, but collecting dense, high-resolution fingerprints during on-site surveys is labor-intensive and incurs high computation/storage costs during matching. Additionally, efficient fingerprint extraction often constrains users to specific shooting poses, with deviations markedly reducing localization accuracy. To address these challenges, we introduce ARGILS, an Automated Real-time Generative Image Localization System. The key idea is to use cross sparse sampling instead of dense sampling, generate fingerprint features for missing locations, and quickly match locations through feature orthogonal decomposition. Cross sparse sampling ensures full coverage of scene features and helps to generate missing fingerprints. To maintain high localization resolution with sparse sampling, we designed a distance-constrained generative adversarial network to generate fingerprints for unsampled locations. Additionally, we developed an orthogonal fingerprint extraction method to decompose image features into horizontal and vertical directions in 2D space. To improve robustness against obstacles, we implemented a scanning localization scheme using key frame filtering and clustering. We have implemented ARGILS and performed extensive real-world evaluations. Experiment results show that when reducing 75% site survey effort, the average location error of ARGILS is around 2.5m in a shopping mall, 48% higher than state-of-the-art methods. ARGILS can also efficiently speed up localization process, with the time consumption ranging from 0.1 to 0.3 seconds on smartphones of various configurations.

Using multiple Unmanned Aerial Vehicles (UAVs) in video surveillance greatly enhances real-time monitoring of large areas. However, images captured by UAVs are separate and limited in view, making stitching crucial for a comprehensive perspective. Current methods combine sensor and visual modalities for video stitching but face challenges in robustness and real-time performance. Environmental disturbances increase errors in sensor and visual data, reducing accuracy and stability, while single-frame stitching incurs high computational costs and slow speeds. To address these issues, we propose SkyEye , a real-time video stitching method for UAV surveillance systems based on multi-modal perception. Specifically, we design a mutual verification method to assess the quality of visual and inertial data. Frames with higher confidence are prioritized for precise stitching. To reduce computational costs, we employ a reference frame scheme that reuses the perspective transformation and feature points of the reference frame for subsequent frames. Meanwhile, we design a video stitching framework based on a per-frame parallel unit to speed up the algorithm execution. We have implemented a prototype of SkyEye and carried out extensive evaluations. Experiment results show that SkyEye outperforms state-of-the-art methods, improving speed by 66.1% and accuracy by 28.2%.

In edge computing, end devices (EDs) containerize tasks with the necessary resources and offload subsets to a nearby high-capacity edge server (ES) to improve efficiency. Most existing research focuses on inseparable task offloading to minimize response times or resource allocation to reduce energy consumption. However, task execution can be speeded up with excessive computing and network resources, it will increase energy consumption and incur unnecessarily high costs. Besides, complex applications like autonomous driving often partition sequential tasks to improve performance, necessitating a joint optimization of sequential task offloading and multi-resource allocation. In this paper, we introduce a Stackelberg game-based framework to model the interplay between these elements.EDs, acting as leaders, determine the offloading breakpoints of sequential tasks and the locality for processing. TheES, as the follower, uses the Karush-Kuhn-Tucker (KKT) conditions and a Boundary-constrained quasi-Particle Swarm Optimization (Bc-qPSO) algorithm to refine computing and network resource allocation, aiming to reduce system costs effectively. Our simulations show that the proposed algorithms reduce cost by approximately 10%-20% compared to traditional methods, highlighting their potential for improving the efficiency of edge computing systems.

The rapid growth in the number of vehicles and types of services such as video transmission improves the quality-of-service (QoS) requirements and increases the difficulty in resisting eavesdropping attacks on the Internet of Vehicles (IoV). Existing video transmission schemes that either ignore the impact of eavesdropping attacks or have the full knowledge of the attack model have performance degradation in highly dynamic IoV systems. In this paper, we propose a reinforcement learning-based secure video transmission scheme for IoV systems, which jointly optimizes the access control policy for each vehicle (i.e., the selection of access nodes such as the base stations or unmanned aerial vehicles) and the corresponding transmit power level against active eavesdropping. This scheme uses the QoS and eavesdropping rate as the criteria to evaluate the long-term risk of each state-action pair, which is estimated by a designed deep Q-network to avoid the risky access control policies that cause severe data leakage or video transmission failure. Simulation results show that our scheme reduces the energy consumption, transmission latency, and eavesdropping rate compared with the benchmark.

In dynamic edge environments, many nodes are constantly changing their geographical locations, leading to unstable network links and even disconnections among nodes relying on wireless communication. Existing research mainly considers optimizing resource allocation and task processing delays, ignoring the impact of transmission delays in fluctuating network environments. However, the transmission time of tasks or requests may be longer than the processing time, which can significantly affect the quality of service for delay-sensitive services. In such scenarios, the static strategies of EdgeMesh always perform terribly under unstable network. This makes the transmission time of task data fluctuate frequently, resulting in an increase in response time. Therefore, we proposed an adaptive strategy based on network-aware to improve the transmission delay. We implemented such a strategy by developing a proxy named NAP to assist EdgeMesh in dynamic traffic management. In order to adapt to the complex and volatile conditions of edge networks, NAP monitors every node in real-time. It determines the feasible path with the shortest expected transmission time based on the size of different task data, network and resources status among nodes, then proxies the traffic and forwards it in the kernel state. Numerous experimental results demonstrate that NAP can reduce task transmission time by $50 \%$ and significantly improve the end-to-end request latency in edge computing environments compared to native strategies of EdgeMesh.

Ultra-high precision motion sensing leveraging computer vision (CV) is a key technology in many high-precision AR/VR applications such as precise industrial manufacture and image-guided surgery, yet conventional CV can be challenged by moiré-based sensing mechanism, thanks to moiré pattern's high sensitivity to six degrees of freedom (6-DoF) pose changes. Unfortunately, existing moiré-based solutions, in their infancy, cannot deal with complicated curvilinear moiré patterns caused by various perspective angles. In this paper, we propose a generalized moiré-based mechanism, MoiréVision, towards practical adoptions; it relies on high-frequency gratings as visual marker to help extract the fine-grained feature points for ultra-high precision motion sensing. As the foundation of general moiré-based sensing, we propose a formulation to characterize "uncontrolled" curvilinear moiré patterns in practical scenarios. To deal with the problem of moiré feature interference in practice, we propose a Gabor-based algorithm to separate overlapped curvilinear moiré patterns from two dimensions. Furthermore, to extract fine-grained feature points for high-precision motion sensing, we propose a bending function-based model and a resolution-enhanced strategy to reconstruct detailed texture of moiré markers and extract moiré feature points at sub-pixel level. Extensive experimental results show that MoiréVision greatly enhances the usability and generalizability of moiré-based sensing systems in real-world applications.

In the realm of AR applications and particularly camera-to-screen interactions, camera tracking stands as a crucial technology. However, the ever-increasing demand for tracking accuracy makes it essential to explore a six-degrees of freedom (6-DoF) tracking technology with ultra-high precision to facilitate micro-motion sensing. In this paper, we propose a novel sensing method MoiréTracker to achieve camera’s 6-DoF pose tracking with ultra-high precision. MoiréTracker outputs camera’s continuous 3-DoF trajectory and 3-DoF posture changes according to the captured moiré patterns, which can be produced by the superposition of camera’s Color Filter Array (CFA) and the projection of screen raster on the CFA plane. Thanks to moiré pattern’s high sensitivity to 6-DoF motions, we characterize the relationship between moiré features and camera’s micro pose changes, so as to realize the continuous 6-DoF pose tracking for camera with ultra-high precision. Moreover, our proposal involves a thumbnail-based method aimed at expanding the working range of MoiréTracker, enabling the pervasive camera-to-screen interactions. We implement a prototype system and evaluate its performance in real-world environments. Extensive experiment results show that MoiréTracker achieves the average trajectory error of 1.20 cm and the posture error of 1.07°.

With the rising of demands for novel Human-Computer Interaction (HCI) approaches in the 3D space, a number of intelligent approaches have been proposed to achieve the HCI by tracking the translation and rotation of the target devices. In this paper, we propose to realize a light-weight, battery-free, 3D motion tracking solution by leveraging a spinning linearly polarized antenna to track a passive RFID tag array. Instead of using the fixed antennas, which can only receive stable signal in some specific environments due to the unpredictable multipath effect, we propose to mitigate the multipath effect and the ambient interference by continuously spinning a linearly polarized antenna, and then extract the most distinctive features based on the optimal reading conditions of the spinning antenna. In particular, because the phase variation around the matching direction is more stable while the RSSI variation around the mismatching direction is more distinctive, we leverage such matching/mismatching property of the linearly polarized antenna to extract the most distinctive features for motion tracking. To depict the property, we build a theoretical model to explain the RSSI and the phase variation of the RFID tag along with the spinning of the antenna, and further extend the model from a single RFID tag to an RFID tag array. Based on the model, we can extract the distinctive RSSI features for the rotation tracking and the stable phase features for the translation tracking. Moreover, to tackle the low rate of feature extraction due to the spinning of antenna, we further propose to enhance the unstable phase features based on the overall trend of other tags with interpolation, such that the sampling rate can be efficiently improved. Finally, we propose a LSTM (Long Short Term Memory)-based network to track the 3D motion based on the signal features extracted based on the polarization model. The experimental results show that our system can achieve an average error of 10.45 cm in the translation tracking, and an average error of$6.02^\circ$in the rotation tracking in the 3D space.

Nowadays, to realize the intelligent manufacturing in Industrial Internet of Things (IIoT) scenarios, novel approaches in computer vision are in great demand to tackle the new challenges in IIoT environment. These approaches, which we callIndustrial Vision, are expected to offer customized solutions for intelligent manufacturing in an accurate, time efficient and robust manner. In this paper, we propose a novel approach to industrial vision, calledEdge-Eye, to rectify the edge deviation automatically for Irradiated Cross-linked Polyethylene Foam (IXPE) production with millimeter-level accuracy. IXPE has been one of the most commonly used materials in industry. During the production process of IXPE sheets, their edges need keep aligned strictly, otherwise, they could quickly get out of the border of the rolling plate and cause the huge economic loss. We deploy a commercial camera with mobile edge node in front of the IXPE sheet to continuously detect and rectify the edge deviation. Particularly, to handle the complex production environment when extracting the edge of IXPE sheet, we deploy a pair of reference bars with high-contrast colors to efficiently differentiate the sheet edge from the background. Then, we propose aBi-direction Edge Tracking methodto perform the edge detection from both vertical and horizontal aspects. To realize the rectification using mobile edge nodes with limited computing resources, we reduce the cost of computation by extracting theMinimized Region of Interest, i.e., the edge area overlapped with the higher contrast reference bar on both sides. We further design a negative feedback control system with multi-stage feedback regulation mechanism, keeping the edge deviation withinmillimeter-level. We implementedEdge-Eyeon the ARM64 platform and performed evaluation in the practical IXPE production process. The experimental results show thatEdge-Eyeachieves the average accuracy of 5 mm for the edge deviation rectification, with the average latency of 200 ms for edge deviation detection. During the process of 20-month real deployment for 36 production lines, 66 manpower per day (90% of the overall manpower) has been saved, and the utilization rate of IXPE material increases from 87% to 94%.

In industrial production, the orientation of facility components can indicate whether the facility is on a regular operating track. For example, when a component gets loose, the orientation variation of the component would exceed the normal range. A common approach for orientation measurement is to attach an inertial measurement unit (IMU) to the target device. However, the IMU requires additional power maintenance. This article presents LightGyro, a cheap and efficient batteryless scheme to measure the orientation, in which we attach a reflective film to the target device and use a camera to capture the light spot on the reflective film. The basic idea of LightGyro is to extract the light spots in the captured frame and use their pixel coordinates to infer the orientation. It is difficult to recognize a single light spot, because the spot lacks distinctive features. To solve the problem, we switch light sources on and off to regulate the appearance of light spots and utilize frame subtraction to extract light spots. The depth of field of light spot is lost in the process of camera projection, which is necessary for the orientation measurement. To address the issue, we propose a light array-based reflection model to extract the depth of field from the relative positions of multiple light spots. To the best of our knowledge, this is the first work to utilize reflection to measure orientation. Experiment results show that the orientation error of LightGyro decreases with the increasing length of the reflection route and the orientation error can achieve less than 1 ˆ .

Handheld devices such as smartphones and cameras often produce shaky videos due to reasons such as hand tremors and movement. Current video stabilization methods mainly estimate camera motion based on feature points, such as corner points, and then smooth the motion to obtain a stabilized video. However, in reality, most videos are shot in dynamic scenes, where some feature points are extracted from moving objects. They seriously affect the estimation of camera motion, and result in unsatisfactory video stabilization. In this paper, we propose a multi-modal-based video stabilization method for dynamic scenes using video frames and gyroscope. The basic idea is to eliminate dynamic feature as their estimated rotation significantly differs from the actual rotation obtained from the gyroscope data. The experimental results show that our method effectively enhances the quality of shaky videos in dynamic scenes.

Nowadays, authentication systems are usually required to provide continuous, contactless, and non-intrusive services. In this paper, we proposeRF-Badge, a vital sign-based authentication scheme on human subjects to meet the above requirements by using RFID technology. We consider two biometric features with individual diversity to characterize the vital sign of users, including themovement effectfrom respiration and thereflection effectfrom organs, especially the heart. To derive the movement effect from respiration, we build a phase-based geometric model to restore the fine-grained badge moving trace as the feature. To derive the reflection effect from human internal organs, we extract the reflection signal from the original signal and generate the spectrum as the feature. Besides, to deal with the feature deviation in different physical conditions of users, we propose a multi-condition network (MCNet) to further guarantee the generalization of RF-Badge. We implement a prototype system and evaluate the performance in real environments. The experiment results show that our system achieves the average false positive rate (FPR) of 3.9 percent and false negative rate (FNR) of 3.3 percent for continuous authentication within four signal cycles.

In the context of Industrial Internet of Things (IIoT), RFID technologies have been widely applied to locate or track tagged objects for achieving item-level intelligence. However, prior localization work encounters two main issues. First, the phase measurement usually contains physical deviation. Existing localization work generally takes the physical center of an RFID antenna as its phase center, which is a key factor in improving localization accuracy but actually different from the physical center in practice. Second, the non-linear localization model is likely to be too complex to run on edge nodes with limited computing resources. In this paper, we present a LInear localizatiON solution, called LION, to perform the phase calibration for antennas with no need for the complex computation nor strong limitations. Specifically, we provide a novel lightweight model to pinpoint the actual antenna position quickly and accurately. Compared to previous localization methods, we reduce the intersection of circles or hyperbolas into radical lines, which greatly reduces the computation cost while guaranteeing the high accuracy. Further, to adapt to the complex environment with various ambient noise and multi-path effect, we leverage the weighted least square method to determine the optimal position. Moreover, we propose an adaptive parameter selection scheme to automatically choose optimal parameters for localization. In this way, LION is able to perform the accurate localization robustly. We implement LION using commercial RFID devices, and evaluate its performance extensively. Experimental results show the necessity of phase calibration as well as the high time efficiency of LION, e.g., the average accuracy improves by 6× and 2.1× for 2D and 3D localization, and the average time consuming is 0.02s and 1.8s for 2D and 3D cases.

Irradiated Cross-linked Polyethylene Foam (IXPE) has been one of the most commonly used materials in industry. During the production process of IXPE sheets, their edges need keep aligned strictly, otherwise, they could quickly get out of the border of the rolling plate and cause the huge economic loss. In this paper, we propose a camera-enabled approach, called Edge-Eye, to rectify the edge deviation automatically for IXPE production with millimeter-level accuracy. We deploy a commercial camera with mobile edge node in front of the IXPE sheet to continuously detect and rectify the edge deviation. Particularly, to handle the complex production en-vironment when extracting the edge of IXPE sheet, we deploy a pair of reference bars with high-contrast colors to efficiently differ-entiate the sheet edge from the background. Then, we propose a Bi-direction Edge Tracking method to perform the edge detection from both vertical and horizontal aspects. To realize the rectification using mobile edge nodes with limited computing resources, we reduce the cost of computation by extracting the Minimized Region of Interest, i.e., the edge area overlapped with the higher contrast reference bar on both sides. We further design a negative feedback control system with multi-stage feedback regulation mechanism, keeping the edge deviation within millimeter-level. We implemented Edge-Eye on the ARM64 platform and performed evaluation in the practical IXPE production process. The experimental results show that Edge-Eye achieves the average accuracy of 5mm for the edge deviation rectification, with the average latency of 200ms for edge deviation detection. During the process of 20-month real deployment for 36 production lines, 66 manpower per day (90% of the overall manpower) has been saved, and the utilization rate of IXPE material increases from 87% to 94%.

Camera tracking has become a key technology for various application scenarios, especially for AR-based camera-to-screen interaction. Demand for subtle motion detection in camera tracking makes it essential to explore the six degrees of freedom (6-DoF) pose detection with ultra-high precision. In this paper, we propose a novel sensing method MoiréPose to achieve ultra-high precision on the camera's 6-DoF pose estimation. The purpose of MoiréPose is to derive the camera's 3-DoF position and 3-DoF posture relative to the screen according to the captured moiré pattern, which is produced by the superposition of the camera's Color Filter Array (CFA) and the screen raster projected onto the CFA layer. Based on moiré pattern's high sensitivity to 6-DoF pose movement and robustness to the environmental interference in the frequency domain, we propose a spectrogram-based method to realize the camera's 6-DoF detection with ultra-high precision. Moreover, we propose a thumbnail-based method to effectively extend the working range of MoiréPose, so as to realize pervasive camera-to-screen interaction. We have implemented a prototype system and evaluate the performance in real-world environments. Extensive experiment results show that MoiréPose achieves an average position error of 7.5mm and an overall posture error of 1.66°.

With the rising of demands for novel human-computer interaction approaches in the 2D plane, a number of intelligent devices come into being. For example, Microsoft Surface Dial supports simple clicks and rotations for the interaction with computer. However, these approaches are dedicated devices, and they might require batteries or have limited functions. In this paper, we propose RF-Dial to realize a light-weight, battery-free and functional 2D human-computer interaction solution via commercial off-the-shelf (COTS) passive RFID tags. What RF-Dial shines is that it can easily turn an ordinary object, e.g., a board eraser, into an intelligent interaction device. By deploying a tag array on the side face of the object together with a dipole tag on the top face, RF-Dial cannot only track the rigid motion of the object but also detect the touch gesture of a user on the surface of the object, including translation, rotation, click, press and hold, and swipe. To do the motion tracking, RF-Dial builds a phase-based model that captures the translation and the rotation of the tagged object simultaneously, by jointly exploiting the information of phase variations and the topology of the tag array. To detect the touch gesture, RF-Dial builds an RSSI-based model that uses the impact of the touching finger on the tag antenna’s impedance to estimate the touch position in real time, which is robust to environmental factors like position or orientation. We implemented a prototype of RF-Dial with commodity RFID devices. Extensive experiments show that RF-Dial achieves an accurate rigid motion tracking, with a small error of 0.6cm for the translation tracking, and a small error of 1.9 degrees for the rotation estimation. Besides, RF-Dial can also detect the touch gesture accurately, as the 90 percent of touch position errors are less than 2.09mm.

Among all the road accidents, speeding is the most deadly factor. To reduce speeding, it is essential to devise efficient schemes for ubiquitous speed monitoring. Traditional approaches either suffers from using special equipment(e.g., radar speed gun) or special deployment(e.g., position-fixed cameras). In this article, we propose SpeedTalker, a mobile phone-based approach to perform speed detection on automobiles. By leveraging the built-in microphones and camera from the mobile phone, SpeedTalker estimates the automobile speed by passively sensing the acoustic and image signals. We propose an integrated solution to effectively estimate the automobile’s speed based on COTS devices, and provide a platform for every pedestrian to help report the speeding event of automobiles. Specifically, we use the time difference of arrivals (TDOA) model based on acoustic signals to figure out the candidate trajectories of automobile, and use the pin-hole model based on image frames to figure out the vertical distance between the user’s position and the automobile’s trajectory, thus to estimate the unique trajectory. Combined with the time stamp of the trajectory, the automobile speed can be estimated. Besides, we propose a method to effectively mitigate the influence of the movement jitters of mobile phone. We implemented a system prototype for SpeedTalker and estimated the automobile speed with high accuracy. Experiment results show that in the scenario of single automobile, SpeedTalker can achieve an average estimation error of 6.1 percent compared to radar speed guns. In the scenario of multiple automobiles, SpeedTalker can achieve an average estimation error of 9.8 percent, which is acceptable for usage.

Nowadays, detecting and evaluating the internal structure of packages becomes a crucial task for logistics systems to guarantee reliability and security. However, prior solutions such as X-ray diffraction and WiFi-based detection are not suitable for this purpose. X-ray-based methods usually require manual analysis or image processing algorithms with high complexity, while WiFi-based solutions may fail to detect complex structures due to the significant error of the RF-signal features. In this article, we propose RF-Detector, a low-cost RFID solution for performing three-dimensional (3D) structure detection of items contained in the packages, including the item orientations and relative locations. We thoroughly investigate a brand-new sensing model for RFID-based 3D structure detection, i.e., revolving scanning. We propose not only the fundamental revolving model but also a novel calibration method for the undesired deployments. We have implemented a prototype system to evaluate the performance of RF-Detector. Extensive evaluations in real settings show the effectiveness of RF-Detector, achieving very high accuracy of the internal 3D structure detection.

Currently, the logistic industry has introduced 3D reconstruction to monitor the package placement in the warehouse. Previous 3D reconstruction solutions mainly utilize computer vision or sensor-based methods, which are restricted to the line-of-sight or the battery life. Therefore, we propose a passive RFID-based solution, called RF-3DScan, to perform 3D reconstruction on tagged packages, including the package orientation and the package stacking. The basic idea is that a moving antenna can obtain RF-signals from the tags attached on packages with the 1D linear mobile scanning. Through extracting phase differences to build angle profiles for each tag, RF-3DScan derives their relative positions, further determines the package orientation and the coarse-grained package stacking. By simply performing the 2D scanning, RF-3DScan can provide the fine-grained package stacking determination. We implement a prototype system of RF-3DScan and evaluate its performance in real settings. Our experiment results show that RF-3DScan can achieve about 92.5 percent identification accuracy of the bottom face, and an average error about 4.08° of thethe rotation angle. For the package stacking, 1D scanning can achieve the comparable performance in comparison with 2D scanning.

Nowadays, detecting and evaluating the internal structure of packages becomes a crucial task for logistics systems to guarantee the reliability and security. However, prior solutions such as X-ray diffraction and WiFi-based detection are not suitable for this purpose. X-ray-based methods usually require manual analysis or image processing algorithms with high complexity, while WiFi-based solutions may fail to detect complex structures due to the significant error of the RF-signal features. In this paper, we propose RF-Detector, a low-cost RFID solution for performing 3D structure detection of items contained in the packages, including the item orientations and relative locations. We thoroughly investigate a brand-new sensing model for RFID-based 3D structure detection, i.e., revolving scanning. We propose not only the fundamental revolving model but also a novel calibration method towards the undesired deployment. We have implemented a prototype system to evaluate the performance of RF-Detector. Extensive evaluations in real settings show the effectiveness of RF-Detector, achieving very high accuracy of the internal 3D structure detection.

Nowadays, the growing demand for the 3D human-computer interaction (HCI) has brought about a number of novel approaches, which achieve the HCI by tracking the motion of different devices, including the translation and the rotation. In this paper, we propose to use a spinning linearly polarized antenna to track the 3D motion of a specified object attached with the passive RFID tag array. Different from the fixed antenna-based solutions, which suffer from the unavoidable signal interferences at some specific positions/orientations, and only achieve the good performance in some feasible sensing conditions, our spinning antenna-based solution seeks to sufficiently suppress the ambient signal interferences and extracts the most distinctive features, by actively spinning the antenna to create the optimal sensing condition. Moreover, by leveraging the matching/mismatching property of the linearly polarized antenna, i.e., in comparison to the circularly polarized antenna, the phase variation around the matching direction is more stable, and the RSSI variation in the mismatching direction is more distinctive, we are able to find more distinctive features to estimate the position and the orientation. We build a model to investigate the RSSI and the phase variation of the RFID tag along with the spinning of the antenna, and further extend the model from a single RFID tag to an RFID tag array. Furthermore, we design corresponding solutions to extract the distinctive RSSI and phase values from the RF-signal variation. Our solution tracks the translation of the tag array based on the phase features, and the rotation of the tag array based on the RSSI variation. The experimental results show that our system can achieve an average error of 13. 6cm in the translation tracking, and an average error of 8.3° in the rotation tracking in the 3D space.

With computer vision-based technologies, current Augmented reality (AR) systems can effectively recognize multiple objects with different visual characteristics. However, only limited degrees of distinctions can be offered among different objects with similar natural features, and inherent information about these objects cannot be effectively extracted. In this paper, we propose TaggedAR, i.e., an RFID-based approach to assist the recognition of multiple tagged objects in AR systems, by deploying additional RFID antennas to the COTS depth camera. By sufficiently exploring the correlations between the depth of field and the received RF-signal, we propose a rotate scanning-based scheme to distinguish multiple tagged objects in the stationary situation, and propose a continuous scanning-based scheme to distinguish multiple tagged human subjects in the mobile situation. By pairing the tags with the objects according to the correlations between the depth of field and RF-signals, we can accurately identify and distinguish multiple tagged objects to realize the vision of “tell me what I see” from the AR system. We have implemented a prototype system to evaluate the actual performance with case studies in a real-world environment. The experiment results show that our solution achieves an average match ratio of 91 percent in distinguishing up to dozens of tagged objects with a high deployment density.

Nowadays, the demand for novel approaches of 2D human-computer interaction has enabled the emergence of a number of intelligent devices, such as Microsoft Surface Dial. Surface Dial realizes 2D interactions with the computer via simple clicks and rotations. In this paper, we propose RF-Dial, a battery-free solution for 2D human-computer interaction based on RFID tag arrays. We attach an array of RFID tags on the surface of an object, and continuously track the translation and rotation of the tagged object with an orthogonally deployed RFID antenna pair. In this way, we are able to transform an ordinary object like a board eraser into an intelligent HCI device. According to the RF-signals from the tag array, we build a geometric model to depict the relationship between the phase variations of the tag array and the rigid transformation of the tagged object, including the translation and rotation. By referring to the fixed topology of the tag array, we are able to accurately extract the translation and rotation of the tagged object during the moving process. Moreover, considering the variation of phase contours of the RF-signals at different positions, we divide the overall scanning area into the linear region and non-linear region in regard to the relationship between the phase variation and the tag movement, and propose tracking solutions for the two regions, respectively. We implemented a prototype system and evaluated the performance of RF-Dial in the real environment. The experiments show that RF-Dial achieved an average accuracy of 0. 6cm in the translation tracking, and an average accuracy of 1.9°in the rotation tracking.

Nowadays, novel approaches of 3D human-computer interaction have enabled the capability of manipulating in the 3D space rather than 2D space. For example, Microsoft Surface Pen leverages the embedded sensors to sense the 3D manipulations, such as inclining the pen to get bolder handwriting. In this paper, we propose RF-Brush, a battery-free and light-weight solution for 3D human-computer interaction based on RFID, by simply attaching a linear RFID tag array onto the linear shaped object like a brush. RF-Brush senses the 3D orientation and 2D movement of the linear shaped object, when the human subject is drawing with this object in the 3D space. Here, the 3D orientation refers to the relative orientation of the linear shaped object to the operating plane, whereas the 2D movement refers to the moving trace in the 2D operating plane. In this way, we are able to transform an ordinary linear shaped object like a brush or pen to an intelligent HCI device. Particularly, we build two geometric models to depict the relationship between the RF-signal and the 3D orientation as well as 2D movement, respectively. Based on the geometric model, we propose the linear tag array-based HCI solution, implemented a prototype system, and evaluated the performance in real environment. The experiments show that RF-Brush achieves an average error of 5.7° and 8.6° of elevation and azimuthal angle, respectively, and an average error of 3.8cm and 4.2cm in movement tracking along X-axis and Y-axis, respectively. Moreover, RF-Brush achieves 89% in letter recognition accuracy.

Infusion monitoring is in great demand for the hospital. In this demo, we propose RF-iCare, an RFID-based approach for monitoring the infusion status, including the liquid level and the drop speed. With a tag array attached on the infusion bottle, we design an RSSI-based signal match model to estimate the liquid level. With a tag attached on the Murphy's dropper, we leverage the phase variation of the tag to estimate the drop speed. We implement RF-iCare with a COTS RFID system and evaluate it in the real-world hospitals. Our experiments demonstrate that RF-iCare can accurately monitor the completion of the infusion over 91% tests, and estimate the liquid level with the mean accuracy of 0.8 cm as well as the drop speed with the error rate less than 3%.

In this paper, we propose 3DLoc, which performs 3-dimensional localization on the tagged objects by using the RFID tag arrays. 3DLoc deploys three arrays of RFID tags on three mutually orthogonal surfaces of each object. When performing 3D localization, 3DLoc continuously moves the RFID antenna and scans the tagged objects in a 2-dimensional space right in front of the tagged objects. It then estimates the object's 3D position according to the phases from the tag arrays. By referring to the fixed layout of the tag array, we use Angle of Arrival-based schemes to accurately estimate the tagged objects' orientation and 3D coordinates in the 3D space. To suppress the localization errors caused by the multipath effect, we use the linear relationship of the AoA parameters to remove the unexpected outliers from the estimated results. We have implemented a prototype system and evaluated the actual performance in the real complex environment. The experimental results show that 3DLoc achieves the mean accuracy of 10cm in free space and 15.3cm in the multipath environment for the tagged object.

Nowadays, 3D reconstruction has been introduced in monitoring the package placement in logistic industry-related applications. Existing 3D econstruction methods are mainly based on computer vision or sensor-based approaches, which are limited by the line-of-sight or battery life constraint. In this paper, we propose RF-3DScan to perform 3D reconstruction on tagged packages via passive RFID, by attaching multiple reference tags onto the surface of the packages. The basic idea is that by moving the antenna along straight lines within a constrained 2-dimensional space, the antenna obtains the RF-signals of the reference tags attached on the packages. By extracting the phase differences to build the angle profile for each tag, RF-3DScan can compare the angle profiles of the different reference tags and derive their relative positions, then further determine the package orientation and stacking for 3D reconstruction. We implement RF- 3DScan and evaluate its performance in real settings. The experiment results show that the average identification accuracy of the bottom face is about 92.5%, and the average estimation error of the rotation angle is about 4.08o.