While Large Language Models (LLMs) demonstrate immense potential for automating integrated circuit (IC) development, their practical deployment is fundamentally limited by restricted context windows. Existing context-extension methods struggle to achieve effective semantic modeling and thorough multi-hop reasoning over extensive, intricate circuit specifications. To address this, we introduce ChipMind, a novel knowledge graph-augmented reasoning framework specifically designed for lengthy IC specifications. ChipMind first transforms circuit specifications into a domain-specific knowledge graph (ChipKG) through the Circuit Semantic-Aware Knowledge Graph Construction methodology. It then leverages the ChipKG-Augmented Reasoning mechanism, combining information-theoretic adaptive retrieval to dynamically trace logical dependencies with intent-aware semantic filtering to prune irrelevant noise, effectively balancing retrieval completeness and precision. Evaluated on an industrial-scale specification reasoning benchmark, ChipMind significantly outperforms state-of-the-art baselines, achieving an average improvement of 34.59% (up to 72.73%). Our framework bridges a critical gap between academic research and practical industrial deployment of LLM-aided Hardware Design (LAD).

Location-based augmented reality (LB-AR) applications, such as Pokemon Go, rely on sub-second GPS updates to deliver responsive and immersive user experiences. However, this high-frequency location reporting introduces serious privacy risks. Unlike traditional Location-Based Services (LBS), LB-AR demands real-time protection under strict latency and quality-of-service (QoS) constraints, while providing strong per-location and trajectory-level privacy guarantees. Existing privacy mechanisms struggle to satisfy these requirements: they either introduce prohibitive latency, significantly degrade application utility, or fail to defend against trajectory inference attacks. To address this challenge, we present PrivAR, the first client-side privacy framework for real-time LB-AR. PrivAR introduces two lightweight mechanisms: (i) Planar Staircase Mechanism (PSM), which uses a staircase-shaped distribution to generate noisy locations with strong per-location privacy, low expected distortion, and minimal computational overhead; and (ii) Planar Staircase Mechanism with Intermediate (PSM-I), an extension of PSM that generates a device-resident intermediate trajectory and selectively reuses previously perturbed outputs when insufficient drift is observed, thereby strengthening trace-level privacy while preserving high QoS. We provide theoretical analysis, extensive evaluation on two public mobility datasets and our GeoTrace dataset, and validate PrivAR in a Pokemon GO-style Android prototype. Results show that PrivAR improves AR QoS (game score) by up to 50% and increases attacker Bayes risk by up to 1.8x, while incurring only 0.06 ms of per-update overhead (less than 0.2% of end-to-end latency).

Large-scale, diverse datasets are essential for training robust learning-based robotic manipulation models; however, their acquisition typically requires controlled environments and specialized hardware in research laboratories. This paper presents VRobotix, a virtual reality (VR)-based framework that enables cost-effective and scalable robotic dataset generation through immersive human-in-the-loop control within a physics-accurate robot simulation. By leveraging off-the-shelf VR headsets (e.g., Oculus Quest 3), VRobotix eliminates the need for physical robots while supporting a URDF-compatible, physics-based simulator that accommodates adaptable robotic platforms and egocentric control interfaces, including handheld controllers and body posture tracking. Benefiting from the physics-based simulation, a unique contribution of VRobotix is the replay module, which can regenerate synchronized multi-modal dataset (kinematic states, RGB-D streams) with multiple dataset formats based on the replayable trajectory, supporting various robotic applications. Additionally, an imitation learning module is developed to train control policies using the data collected by VRobotix. Experiments on three initial tasks—pushing, grasping, and stacking—demonstrate a high data collection success rate, averaging 92.0%. Furthermore, policies trained on just 50 trials achieve a 100% task success rate. VRobotix reduces infrastructure costs while generating ROS-compatible datasets, democratizing scalable robotic data acquisition.

Pouring fluids is a routine task for humans but challenging for high-DoF robots, particularly given fluid simulation’s computational demands while training policies. In this paper, we propose DexPour, a novel reinforcement learning method with hierarchical rewards and Approximated Proxy Abstraction (APA) method. APA efficiently approximates liquid behavior using a small set of spheres, reducing computational overhead. Meanwhile, our hierarchical reward framework breaks down the intricate pouring process into four distinct stages—approach, grasp, transport, and pour—providing fine-grained feedback and fostering stable policy learning. Extensive experiments demonstrate that DexPour achieves a 92% fluid transfer efficiency with a 70% cup fill and a 99% efficiency at 30% fill, highlighting its robust performance across varying liquid volumes. Ablation studies highlight the contribution of each component, confirming the necessity of detailed stage-wise guidance for complex dexterous manipulation. In addition, we compare DexPour with a full fluid simulation baseline, showing comparable pouring efficiency while reducing training time by 81.6%, demonstrating DexPour’s efficiency and practical viability for fluid manipulation tasks.

Tactile sensors can significantly enhance the perception of humanoid robotics systems by providing contact information that facilitates human-like interactions. However, existing commercial tactile sensors focus on improving the resolution and sensitivity of single-modal detection with high-cost components and densely integrated design, incurring complex manufacturing processes and unaffordable prices. In this work, we present Bio-Skin, a cost-effective multi-modal tactile sensor that utilizes single-axis Hall-Effect sensors for planar normal force measurement and bar-shape piezo resistors for 2D shear force measurement. A thermistor coupling with a heating wire is integrated into a silicone body to achieve temperature sensation and thermostatic function analogous to human skin. We also present a cross-reference framework to validate the two modalities of the force sensing signal, improving the sensing fidelity in a complex electromagnetic environment. Bio-Skin has a multi-layer design, and each layer is manufactured sequentially and subsequently integrated, thereby offering a fast production pathway. After calibration, Bio-Skin demonstrates performance metrics—including signal-to-range ratio, sampling rate, and measurement range—comparable to current commercial products, with one-tenth of the cost. The sensor’s real-world performance is evaluated using an Allegro hand in object grasping tasks, while its temperature regulation functionality was assessed in a material detection task.

Dexterous telemanipulation critically relies on the continuous and stable tracking of the human operator’s commands to ensure robust operation. Vison-based tracking methods are widely used but have low stability due to anomalies such as occlusions, inadequate lighting, and loss of sight. Traditional filtering, regression, and interpolation methods are commonly used to compensate for explicit information such as angles and positions. These approaches are restricted to low-dimensional data and often result in information loss compared to the original high-dimensional image and video data. Recent advances in diffusion-based approaches, which can operate on high-dimensional data, have achieved remarkable success in video reconstruction and generation. However, these methods have not been fully explored in continuous control tasks in robotics. This work introduces the Diffusion-Enhanced Telemanipulation (DET) framework, which incorporates the Frame-Difference Detection (FDD) technique to identify and segment anomalies in video streams. These anomalous clips are replaced after reconstruction using diffusion models, ensuring robust telemanipulation performance under challenging visual conditions. We validated this approach in various anomaly scenarios and compared it with the baseline methods. Experiments show that DET achieves an average RMSE reduction of 17.2% compared to the cubic spline and 51.1% compared to FFT-based interpolation for different occlusion durations.

Mobile Virtual Reality (VR) is essential for achieving convenient and immersive human-computer interaction and realizing emerging applications such as Metaverse and spatial computing. However, existing VR technologies require two separate renderings of binocular images, thereby causing a significant bottleneck for mobile devices with limited computing and battery capacity. This paper proposes a new approach to optimizing mobile VR rendering called YORO. By utilizing the per-pixel attribute, YORO can generate binocular VR images from the monocular image through genuinely one rendering, saving half the computation over conventional approaches. Our experimental evaluation and detailed user study indicate that, YORO can save 27% power consumption on average and increase frame rate by 115.2%, while maintaining similar binocular image quality compared with state-of-the-art mobile VR rendering solutions. YORO is production-ready and has already been tested in real VR applications. The source code, demo video, prototype android app, video game engine plugins, and more are released anonymously at YORO-VR.github.io.

The rapid proliferation of Internet of Things (IoT) technologies has revolutionized connectivity but introduced significant cybersecurity vulnerabilities. This study investigates the disparity in IoT security awareness between Computer Science (CS) and non-CS students at the University of Colorado Denver, quantifying the educational gap and its implications. A survey of 300 students (150 CS, 150 non-CS) assessed knowledge on IoT principles, threats, and mitigation strategies. Statistical analysis, including chi-square tests, revealed significant disparities: CS students outperformed non-CS students, with mean scores of 9/10 and 4/10, respectively, highlighting a critical knowledge gap. Results indicate that educational background substantially influences IoT security awareness, underscoring the need for interdisciplinary IoT and cybersecurity education to promote a secure digital ecosystem.

Utilizing unique physiological or behavioral traits, biometrics offers an intuitive authentication approach. However, common biometric modalities are susceptible to ambient factors and privacy concerns. This paper proposes CaphandAuth, a novel capacitive touchscreen-based hand authentication system. Using intrinsic capacitive imaging within the touchscreen, it provides a new secure, cost-effective, and user-friendly biometric authentication solution that is inherently resilient to environmental factors. To this end, CaphandAuth captures consecutive capacitive frames as the hand moves across the touchscreen. These frames are processed with an innovative super-resolution algorithm tailored for deformable objects to enhance details. A learning-based feature extractor then derives expressive and adaptive feature representations from the enhanced images. Extensive experiments demonstrate that CaphandAuth achieves an authentication accuracy of 99.84% and an equal error rate (EER) of 2.77% on a commercial tablet. Moreover, Caphand-Auth exhibits formidable resilience to diverse deceiving attempts, including handprint simulation attacks, counterfeit spoofing attacks, and puppet attacks, making it a robust and secure solution in real-world scenarios.

Skin wounds are often covered with gauze to protect the injury and support the healing process. Accurate wound assessment is essential for monitoring healing progress and guiding treatment decisions. However, existing assessment methods typically require direct exposure of the wound, necessitating the removal of gauze when present. This process disrupts the healing environment and increases the risk of secondary infections. In this paper, we introduce mmSkin, an innovative over-gauze wound assessment system that utilizes millimeter-wave (mmWave) radar technology to evaluate wound characteristics without the need to remove the gauze. Central to this system is the principle that variations in skin moisture, a critical indicator of wound health, significantly influence mmWave signal strength. By analyzing these variations, mmSkin accurately identifies skin moisture levels, thereby enabling precise assessment of wound conditions. To achieve reliable sensing, mmSkin incorporates a denoised mmWave imaging algorithm designed to reduce motion noise and effectively distinguish between signals reflected from the target skin and those from surrounding environmental interference. Additionally, the system integrates a physics-based model to guide the training of its moisture derivation model. This integration ensures that mmSkin can accurately estimate moisture distribution across the wound area, making it a powerful tool for noninvasive wound assessment. Extensive experiments validate the system’s high accuracy in over-gauze wound moisture distribution estimation, achieving a mean moisture error of approximately 0.5% in both wound phantom and invivo tests. Additionally, the system demonstrates a structural similarity index measure (SSIM) of about 0.9 compared to groundtruth moisture distributions in both test scenarios. These results highlight mmSkin’s potential to revolutionize noninvasive wound assessment and improve patient outcomes.

The hand localization problem has been a longstanding focus due to its many applications. The task involves modeling the hand as a singular point and determining its position within a defined coordinate system. However, due to data modality limitations, existing hand localization technologies face several challenges. For example, vision-based localization raises privacy concerns, while wearable-based methods compromise user comfort. In this article, we introduce mmHand, a new device-free, privacy-preserving dynamic hand localization system with pixel-level accuracy, using a single commodity mmWave device. We first propose a mmImage generation tool to fully extract spatial information from raw mmWave data and introduce a novel 2-D image-format representation of mmWave data. Next, we design a framework that provides a new quality evaluation method and pixel space labeling for the mmWave data. Finally, we present a cross-modality spatial feature-enhanced model with high spatial feature extraction capabilities, which can accurately localize hand positions at the pixel level in the mmWave radar U-V pixel coordinate system. We evaluate the system with experiments on 12 subjects in three scenarios, and the results across four metrics demonstrate the effectiveness of our hand localization system.

Pathology image matching is crucial for assisting pathologists in the comprehensive diagnosis of cancerous areas. However, variations in image rotation and staining caused by inherent slide imaging techniques increase the burden on pathologists, complicating the examination of cancer across different pathology slides. To address this challenge, we introduce multi-scale ring rotation-invariant matching (MRRM), which improves image matching efficiency using ring topology, assisting pathologists in robustly aligning biomarker information across various pathology images. Specifically, by employing multi-scale rings as convolution kernels, we accurately locate keypoints from the differencing of the ring pyramid, which not only enhances the likelihood of successful pathology image matching but also supports our feature descriptor in achieving advantageous performance in rotation-invariance. Experiments show that with manually annotated golden landmarks as the standard in 81 cases, exhibiting significantly superior matching accuracy (130.93 $\,\mu \mathrm{m}$) and a success rate of 93.83% compared to other methods, particularly in cases with rotated pathology images. This meets the routine diagnostic requirements of pathologists for cancer diagnosis.

Quantification of information leakage is crucial, especially for privacy-preserving systems such as location-based services (LBS) with integrated privacy mechanisms. Existing quantification mainly utilized two approaches: i) the white-box approach, precise but impractical for complex systems, and ii) the black-box approach, practical but struggles with scalability for large output spaces. Recently, Machine Learning (ML) algorithms have been integrated into the black-box approach to effectively approximate information leakage for independent observations with better scalability. However, this method does not provide precise estimates for dependent observations. Intuitively, once a correlated secret is discovered, it becomes easier for an attacker to predict related secrets, leading to an underestimation of information leakage. This paper introduces an ML-based black-box approach to improve the accuracy of information leakage estimation for systems with correlated data, particularly in trace-based scenarios. Our solution uses an ML model for rough estimation and leverages data correlations to refine inferences for more accurate quantification. Evaluation results from three real-world datasets and one collected dataset confirm our solution's effectiveness in accurately and cost-effectively quantifying system leakage for correlated observations.

Measuring the glucose concentration in liquids is crucial for ensuring the safety of products for individuals with diabetes. This process not only aids in diabetes management but also highlights the importance of taking additional precautions after a diabetes diagnosis. Currently, glucose test strips are widely used to measure the glucose concentration of liquids. It's safe, effective, easy to use, and a cheap alternative to other complex technology with the same use. However, it has sevral setbacks, such as its inability to accurately measure the glucose concentration of cold liquids (0°C - 10°C). This lack of variability can lead to more inconvenience for individuals than benefits. Therefore, in this paper, we propose the usage of millimeter-wave (mmWave) sensing as a contactless, versatile, and easy method to measure glucose levels in cold liquids accurately.

Matching whole slide histopathology images to provide comprehensive information on homologous tissues is beneficial for cancer diagnosis. However, the challenge arises with the Giga-pixel whole slide images (WSIs) when aiming for high-accuracy matching. Learning-based methods are difficult to generalize well with large-size WSIs, necessitating the integration of traditional matching methods to enhance accuracy as the size increases. In this paper, we propose a multi-size guiding matching method applicable high-accuracy requirements. Specifically, we design learning multiscale texture to train deep descriptors, called TDescNet, that trains 64 × 64 × 256 and 256 × 256 × 128 size convolution layer as C64 and C256 descriptors to overcome staining variation and low visibility challenges. Furthermore, we develop the 3D-ring descriptor using sparse keypoints to support the description of large-size WSIs. Finally, we employ C64, C256, and 3D-ring descriptors to progressively guide refined local matching, utilizing geometric consistency to identify correct matching results. Experiments show that when matching WSIs of size 4096 × 4096 pixels, our average matching error is 123.48 μm and the success rate is 93.02 % in 43 cases. Notably, our method achieves an average improvement of 65.52 μm in matching accuracy compared to recent state-of-the-art methods, with enhancements ranging from 36.27 μm to 131.66 μm. Therefore, we achieve high-fidelity whole-slice image matching, and overcome staining variation and low visibility challenges, enabling assistance in comprehensive cancer diagnosis through matched WSIs.

Pattern lock is widely used for user authentication in mobile devices due to its simplicity and ease of remembering. However, it is vulnerable to various attacks,e.g., shoulder surfing attacks. In this paper, we propose FingerPattern, a novel enhanced pattern lock authentication system by using friction sound as a second authentication factor. When the user inputs the pattern by swiping his/her fingertip on the screen, the friction sound is generated based on the user's fingerprint and is unique. Thus, FingerPattern can identify and rule out the illegality by utilizing fingerprint-dependent friction sound even if the adversary has inferred the pattern. By this method, FingerPattern secures pattern lock without the need for a change in user unlocking habits. Extensive experiments demonstrate that FingerPattern can identify legitimate users with 97.5% TAR in one attempt and defend against various attacks (e.g., only 16.8% FAR in five attempts even if the adversary can clearly spy on the user's unlocking process). FingerPattern can be incorporated into the existing pattern lock of mobile devices, which is cost-free. Furthermore, a user experience study shows that FingerPattern is well-received by users.

As automatic speech recognition evolves, deployment of the voice user interface (VUI) has boomingly expanded. Especially since the COVID-19 pandemic, the VUI has gained more attention in online communication owing to its non-contact property. However, the VUI struggles to be applied in public scenes due to the degradation of received audio signals caused by various ambient noises. In this article, we propose Wavoice , the first noise-resistant multi-modal speech recognition system that fuses two distinct voices sensing modalities (i.e., millimeter-wave signals and audio signals from a microphone) together. One key contribution is to model the inherent correlation between millimeter-wave and audio signals. Based on it, Wavoice facilitates the real-time noise-resistant voice activity detection and user targeting from multiple speakers. Additionally, we elaborate on two novel modules for multi-modal fusion embedded into the neural network, leading to accurate speech recognition. Extensive experiments prove the effectiveness of Wavoice under adverse conditions—that is, the character recognition error rate below 1% in a range of 7 m. In terms of robustness and accuracy, Wavoice considerably outperforms existing audio-only speech recognition methods with lower character error and word error rates.

In recent years, the field of distributed deep learning within the Internet of Things (IoT) or the edge has experienced exponential growth. Federated meta-learning has emerged as a significant advancement, enabling collaborative learning among source nodes to establish a global model initialization. This approach allows for optimal performance while necessitating minimal data samples for updating model parameters at the target node. Federated meta-learning has gained increased attention due to its capacity to provide real-time edge intelligence. However, a critical aspect that remains inadequately explored is the recovery of interim meta knowledge’s failure, which constitutes a pivotal key for adapting to new tasks. In this paper, we introduce FMRec, a novel platform designed to offer a fast and flexible recovery mechanism for failed interim meta knowledge in various federated meta-learning scenarios. FMRec serves as a complementary system compatible with different types of federated models and is adaptable to diverse tasks. We present a demonstration of its design and assess its efficiency and reliability through real-world applications.

In autonomous driving, millimeter wave (mmWave) radar has been widely adopted for object detection because of its robustness and reliability under various weather and lighting conditions. For radar object detection, deep neural networks (DNNs) are becoming increasingly important because they are more robust and accurate, and can provide rich semantic information about the detected objects, which is critical for autonomous vehicles (AVs) to make decisions. However, recent studies have shown that DNNs are vulnerable to adversarial attacks. Despite the rapid development of DNN-based radar object detection models, there have been no studies on their vulnerability to adversarial attacks. Although some spoofing attack methods are proposed to attack the radar sensor by actively transmitting specific signals using some special devices, these attacks require sub-nanosecond-level synchronization between the devices and the radar and are very costly, which limits their practicability in real world. In addition, these attack methods can not effectively attack DNN-based radar object detection. To address the above problems, in this paper, we investigate the possibility of using a few adversarial objects to attack the DNN-based radar object detection models through passive reflection. These objects can be easily fabricated using 3D printing and metal foils at low cost. By placing these adversarial objects at some specific locations on a target vehicle, we can easily fool the victim AV's radar object detection model. The experimental results demonstrate that the attacker can achieve the attack goal by using only two adversarial objects and conceal them as car signs, which have good stealthiness and flexibility. To the best of our knowledge, this is the first study on the passive-reflection-based attacks against the DNN-based radar object detection models using low-cost, readily-available and easily concealable geometric shaped objects.

Sign language is a primary language used by deaf and hard-of-hearing (DHH) communities. However, existing sign language translation solutions primarily focus on recognizing manual markers. The non-manual markers, such as negative head shaking, question markers, and mouthing, are critical grammatical and semantic components of sign language for better usability and generalizability. Considering the significant role of non-manual markers, we propose the TransASL, a real-time, end-to-end system for sign language recognition and translation. TransASL extracts feature from both manual markers and non-manual markers via a customized eyeglasses-style wearable device with two parallel sensing modalities. Manual marker information is collected by two pairs of outward-facing microphones and speakers mounted to the legs of the eyeglasses. In contrast, non-manual marker information is acquired from a pair of inward-facing microphones and speakers connected to the eyeglasses. Both manual and non-manual marker features undergo a multi-modal, multi-channel fusion network and are eventually recognized as comprehensible ASL content. We evaluate the recognition performance of various sign language expressions at both the word and sentence levels. Given 80 frequently used ASL words and 40 meaningful sentences consisting of manual and non-manual markers, TransASL can achieve the WER of 8.3% and 7.1%, respectively. Our proposed work reveals a great potential for convenient ASL recognition in daily communications between ASL signers and hearing people.

As the demand for precision medicine rapidly grows, companion diagnostics is proposed to monitor and evaluate therapeutic effects for adjusting medicine plans in time. Although a set of clinical companion diagnostics tools (e.g., polymerase chain reaction) have been investigated, they are expensive and only accessible in a lab environment, which hinders the promotion to broader patients. In light of this situation, we take the first steps towards developing a real-world companion diagnostic tool by leveraging mobile technology. In this paper, we present TherapyPal, a privacy-preserving medicine effectiveness computational framework by harnessing semantic hashing-based digital symptomatic phenotyping. Specifically, sensor data captured from daily-life activities is first transformed into spectrograms. Then, we develop a hashing learning network to extract privacy-masked symptomatic phenotypes on smartphones. Afterward, symptomatic hashes at different medicine states are fed to a contrastive learning network in the cloud for treatment effectiveness detection. To evaluate the performance, we conduct a clinical study among 65 Parkinson's disease (PD) patients under dopaminergic drug treatment. The results show that TherapyPal can achieve around 84.1% medicine effectiveness detection accuracy among patients and above 0.925 privacy-masked scores for protecting each private attribute, which validates the reliability and security of TherapyPal to be used as a real-world companion diagnostics tool.

With the increasing deployment of voice-controlled devices in homes and enterprises, there is an urgent demand for voice identification to prevent unauthorized access to sensitive information and property loss. However, due to the broadcast nature of sound wave, a voice-only system is vulnerable to adverse conditions and malicious attacks. We observe that the cooperation of millimeter waves (mmWave) and voice signals can significantly improve the effectiveness and security of user identification. Based on the properties, we propose a multi-modal user identification system (named WavoID) by fusing the uniqueness of mmWave-sensed vocal vibration and mic-recorded voice of users. To estimate fine-grained waveforms, WavoID splits signals and adaptively combines useful decomposed signals according to correlative contents in both mmWave and voice. An elaborated anti-spoofing module in WavoID comprising biometric bimodal information defend against attacks. WavoID produces and fuses the response maps of mmWave and voice to improve the representation power of fused features, benefiting accurate identification, even facing adverse circumstances. We evaluate WavoID using commercial sensors on extensive experiments. WavoID has significant performance on user identification with over 98% accuracy on 100 user datasets.

With the continuing growth of voice-controlled devices, voice metrics have been widely used for user identification. However, voice biometrics is vulnerable to replay attacks and ambient noise. We identify that the fundamental vulnerability in voice biometrics is rooted in its indirect sensing modality (e.g., microphone). In this paper, we presentVocalPrint, a resilient mmWave interrogation system which directly captures and analyzes the vocal vibrations for user authentication. Specifically,VocalPrintexploits the unique disturbance of the skin-reflect radio frequency (RF) signals around the near-throat region of the user, caused by the vocal vibrations. The complex ambient noise is isolated from the RF signal using a novel resilience-aware clutter suppression approach for preserving fine-grained vocal biometric properties. Afterward, we extract the vocal tract and vocal source features and input them into an ensemble classifier for authentication.VocalPrintis practical as it allows the effortless transition to a smartphone while having sufficient usability due to its non-contact nature. Our experimental results from 41 participants with different interrogation distances, orientations, and body motions show thatVocalPrintachieves over 96 percent authentication accuracy even under unfavorable conditions. We demonstrate the resilience of our system against complex noise interference and spoof attacks of various threat levels.

Earpiece mode of smartphones is often used for confidential communication. In this paper, we proposed a remote(>2m) and motion-resilient attack on smartphone earpiece. We developed an end-to-end eavesdropping system mmEve based on a commercial mmWave sensor to recover speech emitted from smartphone earpiece. The rationale of the attack is based on our observation that, soundwaves emitted from the smartphone's earpiece have a strong correlation with reflected mmWaves from the smartphone's rear. However, we find the recovered speech suffers from the sensor's self-noise and smartphone user's motion which limit attack distance to less than 2m, causing limited threats in real world. We modeled the motion interference under mmWave sensing and proposed a motion-resilient solution by optimizing the fitting function on I/Q plane. To achieve a practical attack with reasonable attack distance, we developed a GAN-based denoising scheme to eliminate the noise pattern of the sensor, which boosted the attack range to 6--8m. We evaluated mmEve with extensive experiments and find 23 different models of smartphones manufactured by Samsung, Huawei, etc. can be compromised by the proposed attack.

With the aging of the population and the long-time wearing of earphones, hearing health has gradually emerged as a worldwide health issue. Early detection of hearing health conditions would greatly reduce potential risks with timely medical intervention. This study proposes an earphone-based ear condition monitoring system, named EarHealth, which is low-cost, non-invasive, and easily usable in daily life. It can detect three major hearing health conditions: ruptured eardrum, earwax buildup and blockage, and otitis media. By analyzing the recorded echoes evoked by a chirp sound stimulus, EarHealth recognizes the distinguishable characteristics from ear canal structure and eardrum mobility. EarHealth achieves an accuracy of 82.6% in 92 human subjects, including 27 normal subjects, 22 patients with ruptured eardrum, 25 patients with otitis media, and 18 patients with earwax blockage. EarHealth is the first earphone-based system capable of monitoring hearing health conditions by utilizing the ear canal geometry and eardrum mobility. It is anticipated that EarHealth would provide pervasive and proactive protection for hearing health.

Voice user interfaces (VUIs) have been adopted in many IoT and mobile devices in daily life. VUIs provide a good user experience with lower-cost hardware (i.e., microphone) and higher throughput (compared with keyboard and touchscreen). Currently, identity authentication and receiving commands are the two most common interactions through VUIs, leaving physiological information in the voice unexploited. Recognizing this untapped potential, we propose VocalHR to extend VUIs beyond voice commands to heart activity sensing without additional hardware. VocalHR is built upon the voice-heart modulation effect, which is rooted in the cardiac activities' impacts on the behavior of the vocal organ during voice production. VocalHR captures voice features of cardiac activity in multiple voice organs and proposes a deep learning pipeline to transform features into cardiac activities. As this is the first study exploring voice-based heart activity sensing, we conducted extensive experiments on 43 demographically diverse subjects to verify the intrinsic link between voice and heart activities. On average, VocalHR can achieve less than 11.1% normalized sensing error on the heart event timing. Our further evaluation shows VocalHR is robust to different microphone specifications and varying speech rates.

Playing is an essential way for preschoolers to learn. There are three types of games for preschoolers: functional play, constructive play, and symbolic play. However, existing works/games can only work for one specific type of children's play. Therefore, we designed and implemented an extensible children's game framework based on Augmented Reality building blocks. We implement an AR prototype from scratch that achieves up to 8 blocks with 81% detection rate and overhead of 21ms (46 FPS) on average.

Virtual Reality (VR) training is an emerging method, which is widely deployed in more and more applications. Compared with traditional physical training and video games-based training, VR training can not only provide a sense of realism and immersion similar to physical training but can also train at any time and place, saving time and money. However, due to some constraints like lacking reflections of the ambient environment, the realism and immersion of VR training are insufficient. Therefore, in this paper, we propose enhanced VR training which senses the ambient environment and reflects them as dynamic unexpected training tasks to solve the above problems.

Temperature sensor is one of the most widespread technologies in the IoT era. Wireless temperature monitoring systems are convenient to deploy and can drive mass applications in the fields of smart home, transportation and logistics. Currently, wireless temperature monitoring products are based on microelectronic and semiconductor components, which are not cost-effective (e.g., a few dollars) and more importantly, generate electronic wastes. In this work, we present ThermoWave, a new paradigm of wireless temperature monitoring that is ecological, battery-less, and ultra-low cost. Specifically, ThermoWave is on the basis of the thermal scattering effect on millimeter-wave (mmWave) signals. Specifically, cholesteryl materials align their molecular patterns at different environmental temperatures, and this temperature-induced pattern change will be modulated and sensed by the scattered mmWave signals. There are three functional modules in the ThermoWave system. The ThermoTag is a cholesteryl material inked film or paper tag that can be conveniently attached to the object of interest to monitor temperature changes. Each ThermoTag costs less than 0.01 dollars. The temperature modulated mmWave scattering will be received by a mmWave-radar based ThermoScanner and demodulated by a software-based temperature decoder ThermoSense, which includes a model-based method (i.e., ThermoDot) for point temperature estimation and a data-driven method (i.e., ThermoNet) for thermal imaging. We prototype and evaluate the ThermoWave system performance in both controlled and real-world setups. Experimental results show that the ThermoWave achieves the precision of ±1.0°F in the range of 30°F to 120°F in a controlled setup. We also investigate the performance in real-world applications, and the ThermoWave can reach the ±3.0°F precision in the temperature estimation. We also test and discuss sustainability, durability, robustness, and cost-effectiveness of the ThermoWave in both design and experiments.

Core body temperature is an important indicator of medical treatment. Sudden changes in core body temperature can be a precursor to neurodegenerative diseases such as Parkinson's disease. These diseases have the potential to strike at any time, therefore, long-term monitoring of core body temperature and alerting to sudden changes in temperature become important. In this paper, we designed an in-ear thermometer to monitor the core body temperature with the help of smartphone.

With the continuing growth of voice-controlled devices, voice metrics have been widely used for user identification. However, voice biometrics is vulnerable to replay attacks and ambient noise. We identify that the fundamental vulnerability in voice biometrics is rooted in its indirect sensing modality (e.g., microphone). In this paper, we present VocalPrint, a resilient mmWave interrogation system which directly captures and analyzes the vocal vibrations for user authentication. Specifically, VocalPrint exploits the unique disturbance of the skin-reflect radio frequency (RF) signals around the near-throat region of the user, caused by the vocal vibrations during communication. The complex ambient noise is isolated from the RF signal using a novel resilience-aware clutter suppression approach for preserving fine-grained vocal biometric properties. Afterward, we extract the text-independent vocal tract and vocal source features and input them to an ensemble classifier for user authentication. VocalPrint is practical as it leverages a low-cost, portable, and energy-efficient hardware allowing effortless transition to a smartphone while having sufficient usability as typical voice authentication systems due to its non-contact nature. Our experimental results from 41 participants with different interrogation distances, orientations, and body motions show that VocalPrint can achieve over 96% authentication accuracy even under unfavorable conditions. We demonstrate the resilience of our system against complex noise interference and spoof attacks of various threat levels.

Digital screens, such as liquid crystal displays (LCDs), are vulnerable to attacks (e.g., "shoulder surfing") that can bypass security protection services (e.g., firewall) to steal confidential information from intended victims. The conventional practice to mitigate these threats is isolation. An isolated zone, without accessibility, proximity, and line-of-sight, seems to bring personal devices to a truly secure place.In this paper, we revisit this historical topic and re-examine the security risk of screen attacks in an isolation scenario mentioned above. Specifically, we identify and validate a new and practical side-channel attack for screen content via liquid crystal nematic state estimation using a low-cost radio-frequency sensor. By leveraging the relationship between the screen content and the states of liquid crystal arrays in displays, we develop WaveSpy, an end-to-end portable through-wall screen attack system. WaveSpy comprises a low-cost, energy-efficient and light-weight millimeter-wave (mmWave) probe which can remotely collect the liquid crystal state response to a set of mmWave stimuli and facilitate screen content inference, even when the victim’s screen is placed in an isolated zone. We intensively evaluate the performance and practicality of WaveSpy in screen attacks, including over 100 different types of content on 30 digital screens of modern electronic devices. WaveSpy achieves an accuracy of 99% in screen content type recognition and a success rate of 87.77% in Top-3 sensitive information retrieval under real-world scenarios, respectively. Furthermore, we discuss several potential defense mechanisms to mitigate screen eavesdropping similar to WaveSpy.

Along with the benefits of Internet of Things (IoT) come potential privacy risks, since billions of the connected devices are granted permission to track information about their users and communicate it to other parties over the Internet. Of particular interest to the adversary is the user identity which constantly plays an important role in launching attacks. While the exposure of a certain type of physical biometrics or device identity is extensively studied, the compound effect of leakage from both sides remains unknown in multi-modal sensing environments. In this work, we explore the feasibility of the compound identity leakage across cyber-physical spaces and unveil that co-located smart device IDs (e.g., smartphone MAC addresses) and physical biometrics (e.g., facial/vocal samples) are side channels to each other. It is demonstrated that our method is robust to various observation noise in the wild and an attacker can comprehensively profile victims in multi-dimension with nearly zero analysis effort. Two real-world experiments on different biometrics and device IDs show that the presented approach can compromise more than 70% of device IDs and harvests multiple biometric clusters with purity at the same time.

Digital devices have become a necessity in our daily life, with digital screens acting as a gateway to access a plethora of information present in the underlying device. However, these devices emit visible light through screens where long-term use can lead to significant screen exposure, further influencing users' health. Conventional methods on screen exposure detection (\textite.g., photo logger) are usually privacy-invasive and expensive, further, require ideal light conditions, which are unattainable in real practice. Considering the light intensity and spectrum vary among different light sources, an effective screen spectrum estimation can provide vital information about screen exposure. To this end, we first investigate the characteristics of the junction between p-type and n-type semiconductor (i.e., PN junction) to sense the spectrum under various conditions. Empirically, we design and implement, \textsfSpecEye, an end-to-end, low cost, wearable, and privacy-preserving screen exposure detection system with a mobile application. For validating the performance of our system, we conduct comprehensive experiments with $54$ commodity digital screens, at $43$ distinct locations, with results showing a base accuracy of $99$%, and an equal error rate (EER) approaching $0.80$% under the controlled lab setup. Moreover, we assess the reliability, robustness, and performance variation of \textsfSpecEye under various real-world circumstances to observe a stable accuracy of $95$%. Our real-world study indicates \textsfSpecEye is a promising system for screen exposure detection in everyday life.

Voice-user interface (VUI) has become an integral component in modern personal devices (\textite.g., smartphones, voice assistant) by fundamentally evolving the information sharing between the user and device. Acoustic sensing for VUI is designed to sense all acoustic objects; however, the existing VUI mechanism can only offer low-quality speech sensing. This is due to the audible and inaudible interference from complex ambient noise that limits the performance of VUI by causing denial-of-service (DoS) of user requests. Therefore, it is of paramount importance to enable noise-resistant speech sensing in VUI for executing critical tasks with superior efficiency and precision in robust environments. To this end, we investigate the feasibility of employing radio-frequency signals, such as millimeter wave (mmWave) for sensing the noise-resistant voice of an individual. We first perform an in-depth study behind the rationale of voice generation and resulting vocal vibrations. From the obtained insights, we presentWaveEar, an end-to-end noise-resistant speech sensing system.WaveEar comprises a low-cost mmWave probe to localize the position of the speaker among multiple people and direct the mmWave signals towards the near-throat region of the speaker for sensing his/her vocal vibrations. The received signal, containing the speech information, is fed to our novel deep neural network for recovering the voice through exhaustive extraction. Our experimental evaluation under real-world scenarios with 21 participants shows the effectiveness ofWaveEar to precisely infer the noise-resistant voice and enable a pervasive VUI in modern electronic devices.

Hidden electronics possess the risk of both security threat and privacy intrusion. We present a wireless hidden electronic recognition system, through electronic components unique mmWave nonlinear responses to identify the threats. We then evaluate E-Eye's performance and robustness with a controlled experiment and a field study using iconic devices and score the system with metrics. Results prove that E-Eye is an accurate and robust hidden electronic recognition system.

Inventory management is pivotal in the supply chain to supervise the non-capitalized products and stock items. Item counting, indexing and identification are the major jobs of inventory management. Currently, the most adopted inventory technologies in product counting/identification are using either the laser-scannable barcode or the radio-frequency identification (RFID). However, the laser-scannable barcode is entangled by an alignment issue (i.e., the laser reader must align with one barcode in line-of-sight), and the RFID is economically and environmentally unfriendly (i.e., high-cost and not naturally disposable). To this end, we propose FerroTag which is a paper-based mmWave-scannable tagging infrastructure for the next generation inventory management system, featuring ultra-low cost, environment-friendly, battery-free and in-situ (i.e., multiple tags can be simultaneously processed outside the line-of-sight). FerroTag is developed on top of the FerroRF effects. Specifically, the magnetic nanoparticles within the ferrofluidic ink reply to probing mmWave with classifiable features (i.e., the FerroRF response). By designating the ink pattern and hence the location of particles, the related FerroRF response can be modified. Thus, a specifically designated ferrofluidic ink printed pattern, which is associated with a unique FerroRF response, is a remotely retrievable (a.k.a., mmWave-scannable) identity. Furthermore, we augment FerroTag by designing a high capacity pattern system and a fine-grained identification protocol such that the capacity and robustness of FerroTag can be systematically improved in mass product management in inventory. Last but not least, we evaluate the performance of FerroTag with 201 different tag design patterns. Results show that FerroTag can identify tags with an accuracy of more than 99% in a controlled lab setup. Moreover, we examine the reliability, robustness and performance of FerroTag under various real-world circumstances, where FerroTag maintains the accuracy over 97%. Therefore, FerroTag is a promising tagging infrastructure for the applications in inventory management systems.

Continuous authentication is superior to conventional one-pass authentication by maintaining the security level of a system throughout the entire login session. Leveraging the unique geometric and non-volitional credentials in the cardiac motion, we present a trustworthy, continuous, and non-contact user authentication system. Based on a pilot study with 78 subjects, we evaluate Cardiac Scan in terms of accuracy, authentication time, permanence, and vulnerability. The results show that Cardiac Scan is a robust and usable continuous authentication system.

Motor learning is the ability to acquire a new motor skill, which plays an important role in rehabilitation as patients learn exercise programs or modify movements to regain pain free function. In this paper, we design an easy-to-use multimodal sensor system to assess motor learning. We developed a motor learning assessment device with a touch screen and Leap Motion to record the subject hand movement during a Serial Reaction Time Task(SRTT). The SRTT consists of upper limb reaching to targets in multi-dimensions. The device records metrics of time and movement efficiency and examines motor learning based on data analysis. This device can provide clinicians with data that can inform their approach to training. We recruited a total of 11 participants, with and without chronic pain to evaluate the device using a classifier model to assess participants' performance. The model shows our system works well to identify motor learning differences in individuals with and without chronic pain.

As 3D printing technology begins to outpace traditional manufacturing, malicious users increasingly have sought to leverage this widely accessible platform to produce unlawful tools for criminal activities. Therefore, it is of paramount importance to identify the origin of unlawful 3D printed products using digital forensics. Traditional countermeasures, including information embedding or watermarking, rely on supervised manufacturing process and are impractical for identifying the origin of 3D printed tools in criminal applications. We argue that 3D printers possess unique fingerprints, which arise from hardware imperfections during the manufacturing process, causing discrepancies in the line formation of printed physical objects. These variations appear repeatedly and result in unique textures that can serve as a viable fingerprint on associated 3D printed products. To address the challenge of traditional forensics in identifying unlawful 3D printed products, we present PrinTracker, the 3D printer identification system, which can precisely trace the physical object to its source 3D printer based on their fingerprint. Results indicate that PrinTracker provides a high accuracy using 14 different 3D printers. Under unfavorable conditions (e.g. restricted sample area, location and process), the PrinTracker can still achieve an acceptable accuracy of 92%. Furthermore, we examine the effectiveness, robustness, reliability and vulnerabilities of the PrinTracker in multiple real-world scenarios.

No abstract available.

While malicious attacks on electronic devices (e-devices) have become commonplace, the use of e-devices themselves for malicious attacks has increased (e.g., explosives and eavesdropping). Modern e-devices (e.g., spy cameras, bugs or concealed weapons) can be sealed in parcels/boxes, hidden under clothing or disguised with cardboard to conceal their identities (named as hidden e-devices hereafter), which brings challenges in security screening. Inspection equipment (e.g., X-ray machines) is bulky and expensive. Moreover, screening reliability still rests on human performance, and the throughput in security screening of passengers and luggages is very limited. To this end, we propose to develop a low-cost and practical hidden e-device recognition technique to enable efficient screenings for threats of hidden electronic devices in daily life. First, we investigate and model the characteristics of nonlinear effects, a special passive response of electronic devices under millimeter-wave (mmWave) sensing. Based on this theory and our preliminary experiments, we design and implement, E-Eye, an end-to-end portable hidden electronics recognition system. E-Eye comprises a low-cost (i.e., under $100), portable (i.e., 11.8cm by 4.5cm by 1.8cm) and light-weight (i.e., 45.5g) 24GHz mmWave probe and a smartphone-based e-device recognizer. To validate the E-Eye performance, we conduct experiments with 46 commodity electronic devices under 39 distinct categories. Results show that E-Eye can recognize hidden electronic devices in parcels/boxes with an accuracy of more than 99% and has an equal error rate (EER) approaching 0.44% under a controlled lab setup. Moreover, we evaluate the reliability, robustness and performance variation of E-Eye under various real-world circumstances, and E-Eye can still achieve accuracy over 97%. Intensive evaluation indicates that E-Eye is a promising solution for hidden electronics recognition in daily life.