The scarcity of reliable, extensive datasets hampers the training of effective models for wearable healthcare technology. This data gap frequently introduces biases into training sets, which then carry over into the models themselves. Such inherent biases pose substantial fairness challenges, particularly in sensitive healthcare scenarios. To this end, we propose AFairDNet, an effective active learning framework that utilizes a small collection of annotated data to create an initial classifier, and then continually refines it by incorporating synthesized ‘hard’ signals, representing areas where the model's training is currently insufficient. To ensure both creativity and ethical responsibility in these generated signals, we enhance the signal generation process using Chain of Thought (CoT) reasoning. The model employs real-time iterative CoT refinement of the model's text prompts to condition the multisensor signal diffuser, ensuring that the synthesized multisensor biosignals are not only of high quality but also semantically faithful. Extensive evaluations using two large publicly available multisensor emotion recognition datasets demonstrate that by leveraging a small yet comprehensive collection of synthesized samples (i.e., around 1.4% of the total training set), AFairDNet may boost a baseline classifier's performance, outperforming the state-of-the-art methods. More precisely, in addition to achieving$1.5-3 \%$higher accuracy than current supervised and self-supervised baselines, AFairDNet also boasts an impressive Total Fairness Score, signaling its potential for more responsible and transparent AI-driven synthesized signal generation.

Battery-free sensing has revolutionized IoT applications, but current solutions relying on signal variations between transmitted and backscattered signals remain vulnerable to environmental dynamics and deployment variations. This paper promotes a paradigm shift: inferring targets through antenna impedance variations instead of signal fluctuations, thereby eliminating the impact of unpredictable wireless communication. We demonstrate the effectiveness of this paradigm by reimplementing three existing applications: RIO [1], Keystub [2], and RF-EATS [3]. Compared to original signal-based implementations, our approach shows significant improvements in accuracy and robustness across diverse environments. Furthermore, By integrating antenna engineering with advanced materials science, we also transform antennas into innovative sensors for pressure, temperature, and UV light sensing. This interdisciplinary methodology pushes the boundaries of battery-free sensing, opening new avenues for IoT applications.

Skin wounds are often covered with gauze to protect the injury and support the healing process. Accurate wound assessment is essential for monitoring healing progress and guiding treatment decisions. However, existing assessment methods typically require direct exposure of the wound, necessitating the removal of gauze when present. This process disrupts the healing environment and increases the risk of secondary infections. In this paper, we introduce mmSkin, an innovative over-gauze wound assessment system that utilizes millimeter-wave (mmWave) radar technology to evaluate wound characteristics without the need to remove the gauze. Central to this system is the principle that variations in skin moisture, a critical indicator of wound health, significantly influence mmWave signal strength. By analyzing these variations, mmSkin accurately identifies skin moisture levels, thereby enabling precise assessment of wound conditions. To achieve reliable sensing, mmSkin incorporates a denoised mmWave imaging algorithm designed to reduce motion noise and effectively distinguish between signals reflected from the target skin and those from surrounding environmental interference. Additionally, the system integrates a physics-based model to guide the training of its moisture derivation model. This integration ensures that mmSkin can accurately estimate moisture distribution across the wound area, making it a powerful tool for noninvasive wound assessment. Extensive experiments validate the system’s high accuracy in over-gauze wound moisture distribution estimation, achieving a mean moisture error of approximately 0.5% in both wound phantom and invivo tests. Additionally, the system demonstrates a structural similarity index measure (SSIM) of about 0.9 compared to groundtruth moisture distributions in both test scenarios. These results highlight mmSkin’s potential to revolutionize noninvasive wound assessment and improve patient outcomes.

The hand localization problem has been a longstanding focus due to its many applications. The task involves modeling the hand as a singular point and determining its position within a defined coordinate system. However, due to data modality limitations, existing hand localization technologies face several challenges. For example, vision-based localization raises privacy concerns, while wearable-based methods compromise user comfort. In this article, we introduce mmHand, a new device-free, privacy-preserving dynamic hand localization system with pixel-level accuracy, using a single commodity mmWave device. We first propose a mmImage generation tool to fully extract spatial information from raw mmWave data and introduce a novel 2-D image-format representation of mmWave data. Next, we design a framework that provides a new quality evaluation method and pixel space labeling for the mmWave data. Finally, we present a cross-modality spatial feature-enhanced model with high spatial feature extraction capabilities, which can accurately localize hand positions at the pixel level in the mmWave radar U-V pixel coordinate system. We evaluate the system with experiments on 12 subjects in three scenarios, and the results across four metrics demonstrate the effectiveness of our hand localization system.

Speech recognition for individuals with impairments remains a significant challenge due to atypical speech patterns thatconfound traditional acoustic-only models. This study introduces NeuroSpeech, a novel multimodal framework that integrateselectroencephalography (EEG) with acoustic features to improve recognition accuracy, robustness, and efficiency. A large-scale random search identified optimal EEG encoder configurations and feature extraction parameters, with window size and overlap ($p < 0.001$) emerging as critical factors. Explainable AI (XAI) methods, specifically SHAP, provided insights into model decision-making, supporting interpretability and clinical translation. Evaluations were conducted on two publicly available datasets: Spanish commands and vowels (UNLP-CONICET) and English phonemes and words (KaraOne). Under clean conditions, NeuroSpeech achieved near-perfect accuracy ($F1 = 0.986$ on Spanish; 0.837 on English), while in noisy conditions (SNR = 0.5) it maintained strong performance ($F1 = 0.92$ and 0.70), demonstrating EEG's role as a noise-robust complementary signal. In contrast, Whisper, a state-of-the-art ASR model, showed severe degradation under noise (e.g., $F1$ dropping from 0.81 to 0.46). Finally, complexity analysis showed that NeuroSpeech is lightweight (1-30M parameters) with inference latency of 10-18ms/sample (RTF $< 1$ on CPU and GPU), enabling near-real-time deployment. These results demonstrate NeuroSpeech's significant potential to leverage neural information to augment speech that is compromised, offering a promising advancement for assistive technologies and improved communication for individuals with speech disorders.

We developed an automated photoacoustic and ultrasound breast tomography system that images the patient in the standing pose. The system, named OneTouch-PAT, utilized linear transducer arrays with optical-acoustic combiners for effective dual-modal imaging. During scanning, subjects only need to gently attach their breasts to the imaging window, and co-registered three-dimensional ultrasonic and photoacoustic images of the breast can be obtained within one minute. Our system has a large field of view of 17 cm by 15 cm and achieves an imaging depth of 3 cm with sub-millimeter resolution. A three-dimensional deep-learning network was also developed to further improve the image quality by improving the 3D resolution, enhancing vasculature, eliminating skin signals, and reducing noise. The performance of the system was tested on four healthy subjects and 61 patients with breast cancer. Our results indicate that the ultrasound structural information can be combined with the photoacoustic vascular information for better tissue characterization. Representative cases from different molecular subtypes have indicated different photoacoustic and ultrasound features that could potentially be used for imaging-based cancer classification. Statistical analysis among all patients indicates that the regional photoacoustic intensity and vessel branching points are indicators of breast malignancy. These promising results suggest that our system could significantly enhance breast cancer diagnosis and classification.

Hearing impairment is an increasing global public health concern, yet traditional diagnostic methods are often inaccessible due to their high cost and complexity. This paper introduces a low-cost digital tool for objectively screening hearing disorders, utilizing a novel approach that leverages involuntary pupil dilation in response to sound as a comprehensive measure of auditory processing. The tool is designed for maximum user accessibility, featuring an intuitive interface, a detailed user man-ual, and cost-effective, high-precision sensors. A prototype has been successfully tested in a laboratory setting, demonstrating not only the system's potential for early detection and management of hearing impairments, particularly hyperacusis, but also its reliability and accuracy in capturing subtle auditory responses.

Cardiovascular metrics measurement and monitoring have been a critical need worldwide. The main objective of this work is to prototype an embedded imager for cardiovascular metrics monitoring. Utilizing an 850 nm Near-Infrared (NIR) light source and an Infrared (IR) camera, the system leverages the optical properties of human skin to extract Photoplethysmogram (PPG) signals, heart rate, and vascular structure from video data. We tested the system with 10 participants, comparing its heart rate measurements to those obtained from a contact PPG sensor, achieving an accuracy within ±5 bpm. Additionally, using an artificial hand phantom for blood vessel visualization, the system demonstrated a vessel extraction accuracy with an average error of 10.21 % in blood vessel width, confirming the effectiveness of our NIR-enhanced imaging approach.

Diabetic foot ulcers (DFUs) represent a significant global health challenge for the elderly with high mortality rates and complications. While imaging technologies like NIRS and hyperspectral imaging have improved wound assessment in clinical settings, their cost, and large size limit their use in the home and primary care. On the other hand, existing mobile solutions only capture secondary bio-markers like color and wound size. This paper introduces SigmoidOxy (or σ(Oxy)), a novel smartphone-based perfusion tool for DFU management. SigmoidOxy extracts oxygenation information from standard RGB images captured by smartphone cameras by applying hyperspectral reconstruction models to infer oxygenation. We evaluate SigmoidOxy's performance using the SPECTRALPACA dataset [2] finding an Average Persons R of 0.72 and Average Mean Absolute Error of 0.239 when comparing sigmoid oxygenation signals and analyze its sensitivity to ischemia in the DFUC2021 dataset [17].

In recent years, the WiFi channel state information (CSI) has been increasingly used for human activity recognition (HAR) during activities of daily living, because of nonintrusiveness and privacy preserving properties. However, most previous works require complex processing of CSI signals, and the large number of classification network parameters significantly increases the recognition time and deployment costs. Accordingly, a WiFi signal-based lightweight deep learning (WiLDAR) network is developed in this study to ensure systematic operation on edge computing devices. We combine the random convolution kernel with deep separable convolution and residual structure, so that WiLDAR can easily extract CSI signal features without filtering and denoising. The parameter number and training time of WiLDAR are, thus, much less than those of previous neural networks. In addition, a tiny HAR system using only Raspberry Pi and router is implemented. Experiments verify that WiLDAR can achieve real-time HAR on Internet of Things devices, which makes HAR deployment more convenient. We test WiLDAR on three different fine-grained action data sets to achieve 99%, 93.5%, and 97.5% recognition accuracy, respectively. The demonstrated learning capability of WiLDAR makes it an excellent option for the remote HAR.

IMPORTANCE: Due to insufficient smoking cessation apps for persons living with HIV, our study focused on designing and testing the Sense2Quit app, a patient-facing mHealth tool which integrated visualizations of patient information, specifically smoking use. OBJECTIVES: The purpose of this paper is to detail rigorous human-centered design methods to develop and refine visualizations of smoking data and the contents and user interface of the Sense2Quit app. The Sense2Quit app was created to support tobacco cessation and relapse prevention for people living with HIV. MATERIALS AND METHODS: Twenty people living with HIV who are current or former smokers and 5 informaticians trained in human-computer interaction participated in 5 rounds of usability testing. Participants tested the Sense2Quit app with use cases and provided feedback and then completed a survey. RESULTS: Visualization of smoking behaviors was refined through each round of usability testing. Further, additional features such as daily tips, games, and a homescreen were added to improve the usability of the app. A total of 66 changes were made to the Sense2Quit app based on end-user and expert recommendations. DISCUSSION: While many themes overlapped between usability testing with end-users and heuristic evaluations, there were also discrepancies. End-users and experts approached the app evaluation from different perspectives which ultimately allowed us to fill knowledge gaps and make improvements to the app. CONCLUSION: Findings from our study illustrate the best practices for usability testing for development and refinement of an mHealth-delivered consumer informatics tool for improving tobacco cessation yet further research is needed to fully evaluate how tools informed by target user needs improve health outcomes.

Online voice communications are widely used nowadays. To protect speech from leakage, people tend to initiate the talk in sound-isolated environments. In this paper, we reveal a novel attack that recovers high-quality speech from outside soundproof zones. The rationale of the attack is to leverage sound-sensitive characteristics of piezoelectric materials, i.e., a piezo film that can change the phase of reflected mmWaves when placed in a sound field. If the attacker transmits mmWaves and analyzes reflected signals from the piezo film, the speech information can be compromised. More importantly, the piezo film is paper-like and works without a power supply. We propose a new speech recovery methodology to transform sound waves into wireless signals and build an end-to-end eavesdropping system working as a through-wall “microphone” to recover high-quality speech stealthily. To combat signal attenuation and improve speech quality, we develop a speech-enhancement scheme based on generative adversarial networks and propose to use multi-antenna information for intelligible speech reconstruction. We conduct extensive experiments to evaluate the system. The results indicate that the system achieves over 98% accuracy for digit recognition and works well over 5m away through the wall. We also test the system under complex scenarios and give countermeasures.

Device fingerprints can satisfy the high-security requirement of modern mobile applications (e.g., mobile payments) by guaranteeing the operation is performed on a trusted device. However, existing works on device fingerprints are weak to leakage, which leads to an irreversible failure of the device fingerprint authentication system after suffering from fingerprint theft attacks. The vulnerability drives us to propose a reconfigurable device fingerprint, i.e.,MotoPrint, that can recover the system after suffering from such attacks.MotoPrintstems from the motor vibration that can represent in both signals of the accelerometer and the gyroscope (i.e., they are homologous motion signals). Therefore, we designed a two-path feature extracting network and a sensor-independent training strategy to eliminate sensor noise that can decline authentication performance. In addition,MotoPrinthas a complete reconfiguration mechanism to cope with fingerprint leakage, which brings the damaged authentication system back to health. The evaluation of 80 stand-alone vibration motors and 20 in-built ones shows thatMotoPrintcan achieve high authentication accuracy of 98.5%. Meanwhile, we also demonstrate the reconfiguredMotoPrint, which can also effectively indicate the device's uniqueness with over 98% accuracy, is independent ofMotoPrints under other stimulating codes.

More than 6 million Americans are at risk for Alzheimer's Disease Related Dementias (ADRD), most of whom are 65 or older. The clock drawing test (CDT) is a quick, simple, and effective technique that has the potential advantage of self-management and screening for ADRD patients. Current CDT-based ADRD screening studies focus more on efficacy, involving many handcrafted features, ignoring data modalities, and lacking validation. This paper aims to propose a unified telemedicine framework for fully and semi-automatic effective early ADRD screening based on multimodal and agile data fusion, focusing on the interpretability and validation of the model by using gradient-weighted class activation mapping (Grad-CAM) and locally linear embedding (LLE). The datasets for this work include 1,662 samples of CDT images and related demographic and cognitive information. The fully automatic case involving only CDT images can achieve the highest AUC of 81 with a 75 recall rate in binary screening. The multimodal data fusion in the semi-automatic case can achieve up to 90 AUC with an 83 recall rate. The visualization of the Convolutional Neural Networks (CNNs) shows that it can automatically obtain critical information about the outline, scale, and clock hands from CDT images, and the analysis of structured features shows that the memory test is key to effective ADRD screening.

As automatic speech recognition evolves, deployment of the voice user interface (VUI) has boomingly expanded. Especially since the COVID-19 pandemic, the VUI has gained more attention in online communication owing to its non-contact property. However, the VUI struggles to be applied in public scenes due to the degradation of received audio signals caused by various ambient noises. In this article, we propose Wavoice , the first noise-resistant multi-modal speech recognition system that fuses two distinct voices sensing modalities (i.e., millimeter-wave signals and audio signals from a microphone) together. One key contribution is to model the inherent correlation between millimeter-wave and audio signals. Based on it, Wavoice facilitates the real-time noise-resistant voice activity detection and user targeting from multiple speakers. Additionally, we elaborate on two novel modules for multi-modal fusion embedded into the neural network, leading to accurate speech recognition. Extensive experiments prove the effectiveness of Wavoice under adverse conditions—that is, the character recognition error rate below 1% in a range of 7 m. In terms of robustness and accuracy, Wavoice considerably outperforms existing audio-only speech recognition methods with lower character error and word error rates.

Non-invasive human vital sign detection has gained significant attention in recent years, with its potential for contactless, long-term monitoring. Advances in radar systems have enabled non-contact detection of human vital signs, emerging as a crucial area of research. The movements of key human organs influence radar signal propagation, offering researchers the opportunity to detect vital signs by analyzing received electromagnetic (EM) signals. In this review, we provide a comprehensive overview of the current state-of-the-art in millimeter-wave (mmWave) sensing for vital sign detection. We explore human anatomy and various measurement methods, including contact and non-contact approaches, and summarize the principles of mmWave radar sensing. To demonstrate how EM signals can be harnessed for vital sign detection, we discuss four mmWave-based vital sign sensing (MVSS) signal models and elaborate on the signal processing chain for MVSS. Additionally, we present an extensive review of deep learning-based MVSS and compare existing studies. Finally, we offer insights into specific applications of MVSS (e.g., biometric authentication) and highlight future research trends in this domain.

In autonomous driving, millimeter wave (mmWave) radar has been widely adopted for object detection because of its robustness and reliability under various weather and lighting conditions. For radar object detection, deep neural networks (DNNs) are becoming increasingly important because they are more robust and accurate, and can provide rich semantic information about the detected objects, which is critical for autonomous vehicles (AVs) to make decisions. However, recent studies have shown that DNNs are vulnerable to adversarial attacks. Despite the rapid development of DNN-based radar object detection models, there have been no studies on their vulnerability to adversarial attacks. Although some spoofing attack methods are proposed to attack the radar sensor by actively transmitting specific signals using some special devices, these attacks require sub-nanosecond-level synchronization between the devices and the radar and are very costly, which limits their practicability in real world. In addition, these attack methods can not effectively attack DNN-based radar object detection. To address the above problems, in this paper, we investigate the possibility of using a few adversarial objects to attack the DNN-based radar object detection models through passive reflection. These objects can be easily fabricated using 3D printing and metal foils at low cost. By placing these adversarial objects at some specific locations on a target vehicle, we can easily fool the victim AV's radar object detection model. The experimental results demonstrate that the attacker can achieve the attack goal by using only two adversarial objects and conceal them as car signs, which have good stealthiness and flexibility. To the best of our knowledge, this is the first study on the passive-reflection-based attacks against the DNN-based radar object detection models using low-cost, readily-available and easily concealable geometric shaped objects.

As the demand for precision medicine rapidly grows, companion diagnostics is proposed to monitor and evaluate therapeutic effects for adjusting medicine plans in time. Although a set of clinical companion diagnostics tools (e.g., polymerase chain reaction) have been investigated, they are expensive and only accessible in a lab environment, which hinders the promotion to broader patients. In light of this situation, we take the first steps towards developing a real-world companion diagnostic tool by leveraging mobile technology. In this paper, we present TherapyPal, a privacy-preserving medicine effectiveness computational framework by harnessing semantic hashing-based digital symptomatic phenotyping. Specifically, sensor data captured from daily-life activities is first transformed into spectrograms. Then, we develop a hashing learning network to extract privacy-masked symptomatic phenotypes on smartphones. Afterward, symptomatic hashes at different medicine states are fed to a contrastive learning network in the cloud for treatment effectiveness detection. To evaluate the performance, we conduct a clinical study among 65 Parkinson's disease (PD) patients under dopaminergic drug treatment. The results show that TherapyPal can achieve around 84.1% medicine effectiveness detection accuracy among patients and above 0.925 privacy-masked scores for protecting each private attribute, which validates the reliability and security of TherapyPal to be used as a real-world companion diagnostics tool.

Fingerprint recognition has been a vital security guard for various applications whose vulnerability has been explored by different works. However, previous works on spoofing fingerprint recognition rely on prior knowledge (e.g., photos and minutiae) of the target fingerprint, which fails to implement in practical scenarios. In this paper, we design a fingerprint spoofing attack, namely FingerFaker, to explore the vulnerability of fingerprint recognition, which can spoof automated fingerprint recognition systems (AFRSs) without prior knowledge of target fingerprints. Specifically, we propose a novel concept of "pseudo-minutiae-set" as an effective optimization object and design a two-stage scheme to optimize "pseudo-minutiaeset" leveraging a two-factor evolutionary strategy. In addition, we use a GAN-based training strategy with a minutiae loss function to pre-train a fingerprint generator to map a "pseudo-minutiae-set" into a fingerprint. We use 6342 fingerprint images to verify the performance of FingerFaker on spoofing the open-source AFRS, which shows a high attack success rate (ASR) of 97.78%. Meanwhile, we conduct a realistic case study on commercial off-the-shelf (COTS) AFRS, where FingerFaker also shows 94.22% ASR. Finally, we explore the impact of different conditions to guide the attack and propose countermeasures to mitigate the harm.

An audio system containing loudspeakers and microphones is the fundamental hardware for voice-enabled devices, enabling voice interaction with mobile applications and smart homes. This paper presents MagBackdoor, the first magnetic field attack that injects malicious commands via a loudspeaker-based backdoor of the audio system, compromising the linked voice interaction system. MagBackdoor focuses on the magnetic threat on loudspeakers and manipulates their sound production stealthily. Consequently, the microphone will inevitably pick up malicious sound generated by the attacked speaker, due to the closely packed arrangement of internal audio systems. To prove the feasibility of MagBackdoor, we conduct comprehensive simulations and experiments. This study further models the mechanism by which an external magnetic field excites the sound production of loudspeakers, giving theoretical guidance to MagBackdoor. Aiming at stealthy magnetic attacks in real-world scenarios, we self-design a prototype that can emit magnetic fields modulated by voice commands. We implement MagBackdoor and evaluate it across a wide range of smart devices involving 16 smartphones, four laptops, two tablets, and three smart speakers, achieving an average 95% injection success rate with high-quality injected acoustic signals.

With the increasing deployment of voice-controlled devices in homes and enterprises, there is an urgent demand for voice identification to prevent unauthorized access to sensitive information and property loss. However, due to the broadcast nature of sound wave, a voice-only system is vulnerable to adverse conditions and malicious attacks. We observe that the cooperation of millimeter waves (mmWave) and voice signals can significantly improve the effectiveness and security of user identification. Based on the properties, we propose a multi-modal user identification system (named WavoID) by fusing the uniqueness of mmWave-sensed vocal vibration and mic-recorded voice of users. To estimate fine-grained waveforms, WavoID splits signals and adaptively combines useful decomposed signals according to correlative contents in both mmWave and voice. An elaborated anti-spoofing module in WavoID comprising biometric bimodal information defend against attacks. WavoID produces and fuses the response maps of mmWave and voice to improve the representation power of fused features, benefiting accurate identification, even facing adverse circumstances. We evaluate WavoID using commercial sensors on extensive experiments. WavoID has significant performance on user identification with over 98% accuracy on 100 user datasets.

With the continuing growth of voice-controlled devices, voice metrics have been widely used for user identification. However, voice biometrics is vulnerable to replay attacks and ambient noise. We identify that the fundamental vulnerability in voice biometrics is rooted in its indirect sensing modality (e.g., microphone). In this paper, we presentVocalPrint, a resilient mmWave interrogation system which directly captures and analyzes the vocal vibrations for user authentication. Specifically,VocalPrintexploits the unique disturbance of the skin-reflect radio frequency (RF) signals around the near-throat region of the user, caused by the vocal vibrations. The complex ambient noise is isolated from the RF signal using a novel resilience-aware clutter suppression approach for preserving fine-grained vocal biometric properties. Afterward, we extract the vocal tract and vocal source features and input them into an ensemble classifier for authentication.VocalPrintis practical as it allows the effortless transition to a smartphone while having sufficient usability due to its non-contact nature. Our experimental results from 41 participants with different interrogation distances, orientations, and body motions show thatVocalPrintachieves over 96 percent authentication accuracy even under unfavorable conditions. We demonstrate the resilience of our system against complex noise interference and spoof attacks of various threat levels.

Incompatible protocols and electromagnetic interference obstruct the realization of an everything-connected Internet of Things (IoT) communication network. Our system, Deaf-Aid, utilizes a stealthy speaker-to-gyroscope channel to build robust communication. Compared with existing solutions adopting physical covert channels, Deaf-Aid is free from the limitations of manual receiver distinction, additional hardware, conditional placement, or physical contact. It exploits ultrasounds to force gyroscopes embedded in receivers to resonate, so as to convey information. We investigate the relationship among axes in a gyroscope to deal with frequency offset and support multi-channel communication. Meanwhile, receivers are identified automatically via device fingerprints consisting of diversity of gyroscopes’ resonant frequency ranges. Furthermore, we enable Deaf-Aid the capability of mobile communication, which is an essential demand for IoT devices. We address the challenge of recovering accurate signals from motion interference. Extensive evaluations, including that on the commercial off-the-shelf devices, demonstrate that Deaf-Aid yields 47 bps with BER below 1%. To our best knowledge, Deaf-Aid is the first work to enable stealthy mobile IoT communication based on inertial sensors.

More and more people turn to online voice communication with loudspeaker-equipped devices due to its convenience. To prevent speech leakage, soundproof rooms are often adopted. This paper presents mmPhone, a novel acoustic eavesdropping system that recovers loudspeaker speech protected by soundproof environments. The key idea is that properties of piezoelectric films in mmWave band can change with sound pressure due to the piezoelectric effect. If the property changes are acquired by an adversary (i.e., characterizing the piezoelectric effect with mmWaves), speech leakage can happen. More importantly, the piezoelectric film can work without a power supply. Base on this, we proposed a methodology using mmWaves to sense the film and decoding the speech from mmWaves, which turns the film into a passive "microphone". To recover intelligible speech, we further develop an enhancement scheme based on a denoising neural network, multi-channel augmentation, and speech synthesis, to compensate for the propagation and penetration loss of mmWaves. We perform extensive experiments to evaluate mmPhone and conduct digit recognition with over 93% accuracy. The results indicate mmPhone can recover high-quality and intelligible speech from a distance over 5m and is resilient to incident angles of sound waves (within 55 degrees) and different types of loudspeakers.

Earpiece mode of smartphones is often used for confidential communication. In this paper, we proposed a remote(>2m) and motion-resilient attack on smartphone earpiece. We developed an end-to-end eavesdropping system mmEve based on a commercial mmWave sensor to recover speech emitted from smartphone earpiece. The rationale of the attack is based on our observation that, soundwaves emitted from the smartphone's earpiece have a strong correlation with reflected mmWaves from the smartphone's rear. However, we find the recovered speech suffers from the sensor's self-noise and smartphone user's motion which limit attack distance to less than 2m, causing limited threats in real world. We modeled the motion interference under mmWave sensing and proposed a motion-resilient solution by optimizing the fitting function on I/Q plane. To achieve a practical attack with reasonable attack distance, we developed a GAN-based denoising scheme to eliminate the noise pattern of the sensor, which boosted the attack range to 6--8m. We evaluated mmEve with extensive experiments and find 23 different models of smartphones manufactured by Samsung, Huawei, etc. can be compromised by the proposed attack.

In this paper, we propose a transformative solution that uses a low-cost light sensor and commodity smartphone to support fast self-assessment of blood perfusion of ulcer regions in daily life. By harnessing the knowledge of light polarization, our system can "see-through" the skin to quantify the spatio-temporal properties of subdermal vasculature in terms of pulsation and hemoglobin. Our evaluation results show that our system can achieve 78.6% accuracy to detect poor and good blood perfusion.

Smart manufacturing and Industry 4.0 are bringing disruptive changes to the manufacturing sector. Smart manufacturing increases productivity, creates safer conditions for workers, and simplifies product customization, all while decreasing business expenses [2]. To this end, we have created a flexible three-component architecture for remote machine management, using it to build a digital twin prototype of a Creality Ender-3 Pro 3D printer located on the University at Buffalo campus. This twin provides users with the ability to monitor and control the machine from anywhere in the world through a web interface. Our system improves upon existing technologies, such as Octoprint [1], through the addition of twin views. It also relies upon cheaper components, using the Arduino and ESP32 rather than the Raspberry Pi. Finally, existing technologies tend to focus on one specific type of machine. In contrast, our framework is flexible, capable of supporting many different machines.

Voice user interfaces (VUIs) have been adopted in many IoT and mobile devices in daily life. VUIs provide a good user experience with lower-cost hardware (i.e., microphone) and higher throughput (compared with keyboard and touchscreen). Currently, identity authentication and receiving commands are the two most common interactions through VUIs, leaving physiological information in the voice unexploited. Recognizing this untapped potential, we propose VocalHR to extend VUIs beyond voice commands to heart activity sensing without additional hardware. VocalHR is built upon the voice-heart modulation effect, which is rooted in the cardiac activities' impacts on the behavior of the vocal organ during voice production. VocalHR captures voice features of cardiac activity in multiple voice organs and proposes a deep learning pipeline to transform features into cardiac activities. As this is the first study exploring voice-based heart activity sensing, we conducted extensive experiments on 43 demographically diverse subjects to verify the intrinsic link between voice and heart activities. On average, VocalHR can achieve less than 11.1% normalized sensing error on the heart event timing. Our further evaluation shows VocalHR is robust to different microphone specifications and varying speech rates.

Monitoring the changes in gait patterns is important to individuals’ health. Gait analysis should be taken as early as possible to prevent gait impairments and improve gait quality. Accurate stride-length estimation and gait rehabilitation activity recognition are fundamental components in gait monitoring, gait analysis, and long-term gait care. This article proposes a novel multimodality deep learning architecture to investigate the applications of stride length (SL) estimation and rehabilitation activity recognition. In order to verify this architecture, we have conducted the data collection and data labeling with our customized wearable sensing system. The sensing system can provide sensor readings from 96 sensors-based pressure array and 3-channels accelerometer and gyroscope. Many experiments with multiple perspective analysis are implemented to evaluate the models’ precision, robustness, and reliability. The multimodality deep learning architecture can map multiple sensor readings to the resulting SL with a mean absolute error of 3.89 cm and accurately detect the gait activity with an accuracy of 97.08%. It correlates the step length estimation and gait activity recognition to fulfill comprehensive long-term gait information statistic. The proposed applications’ implementation enriched our previous gait study and brought insights for clinically relevant wearable gait monitoring and gait analysis.

Smoking cessation is a significant challenge for many people addicted to cigarettes and tobacco. Mobile health-related research into smoking cessation is primarily focused on mobile phone data collection either using self-reporting or sensor monitoring techniques. In the past 5 years with the increased popularity of smartwatch devices, research has been conducted to predict smoking movements associated with smoking behaviors based on accelerometer data analyzed from the internal sensors in a user's smartwatch. Previous smoking detection methods focused on classifying current user smoking behavior. For many users who are trying to quit smoking, this form of detection may be insufficient as the user has already relapsed. In this paper, we present a smoking cessation system utilizing a smartwatch and finger sensor that is capable of detecting pre-smoking activities to discourage users from future smoking behavior. Pre-smoking activities include grabbing a pack of cigarettes or lighting a cigarette and these activities are often immediately succeeded by smoking. Therefore, through accurate detection of pre-smoking activities, we can alert the user before they have relapsed. Our smoking cessation system combines data from a smartwatch for gross accelerometer and gyroscope information and a wearable finger sensor for detailed finger bend-angle information. We compare the results of a smartwatch-only system with a combined smartwatch and finger sensor system to illustrate the accuracy of each system. The combined smartwatch and finger sensor system performed at an 80.6% accuracy for the classification of pre-smoking activities compared to 47.0% accuracy of the smartwatch-only system.

Due to the premise of uniqueness and acceptance, fingerprint has been the most adopted biometric technologies in high-impact applications (e.g., smartphone security, monetary transactions and international-border verification). Although there are an array of commercial fingerprint scanners across different sensing modalities including optical, capacitive, thermal and ultrasonic, existing fingerprint technologies are vulnerable to spoofing attacks via fake-finger in Kanget al., 2003. In this paper, we investigate a new dimension of fingerprint sensing based on the friction-excited sonic wave (in simpler words, ”voice of fingerprint”) from a user swiping his fingertip on everyday surfaces. Specifically, we developSonicPrintto leverage the intrinsic fingerprint ridge information in sonic wave for user identification. First, the complex ambient noise is isolated from the sonic wave using background isolation and adaptive segmentation models. Afterward, a series of multi-level friction descriptors that highlight the target fingerprint information is extracted. These descriptors are fed to a specially designed ensemble classifier for user identification.SonicPrintis practical as it leverages in-built microphones in smart devices, requiring no hardware modifications. As the first exploratory study, our experimental results with 31 participants over three different swipe actions on 12 different types of materials show up to a 98 percent identification accuracy.

Linear-array-based photoacoustic tomography has shown broad applications in biomedical research and preclinical imaging. However, the elevational resolution of a linear array is fundamentally limited due to the weak cylindrical focus of the transducer element. While several methods have been proposed to address this issue, they have all handled the problem in a less time-efficient way. In this work, we propose to improve the elevational resolution of a linear array through Deep-E, a fully dense neural network based on U-net. Deep-E exhibits high computational efficiency by converting the three-dimensional problem into a two-dimension problem: it focused on training a model to enhance the resolution along elevational direction by only using the 2D slices in the axial and elevational plane and thereby reducing the computational burden in simulation and training. We demonstrated the efficacy of Deep-E using various datasets, including simulation, phantom, and human subject results. We found that Deep-E could improve elevational resolution by at least four times and recover the object's true size. We envision that Deep-E will have a significant impact in linear-array-based photoacoustic imaging studies by providing high-speed and high-resolution image enhancement.

The campus environments have traditionally revolved around the use of sustainable and practical mobility vehicles such as bicycles, but similar to pedestrians and bicyclists, the students riding smart-scooter are also vulnerable road users and to severe injuries during road accidents. In this paper, we created a “smart android system”. STEADi, for monitoring the Smart scooter riders. The system uses a Wearable Gait Lab for, a wearable underfoot force-sensing intelligent unit, as one of the main components. The purpose of this system is to help students who are new to using smart scooters on campus to avoid injuries and accidents by alerting the rider about unforeseen conditions. The system provides adequate data for path tracking, Potholes Detection system, and human balancing ability for the Smart Scooter riders. After careful selection of training data, we have been able to integrate a pothole detector system that identifies worse road segments as having potholes. The proposed system is evaluated based on four balance tests on different terrain and with different diverse riding experiences related to the Smart Scooters. The system testing showed that it can successfully detect several real potholes in and around the Cleveland area and is successfully able to alert the riders, including the lesser experienced ones while riding on different terrains for the potential road-related threats.

With the advance in automatic speech recognition, voice user interface has gained popularity recently. Since the COVID-19 pandemic, VUI is increasingly preferred in online communication due to its non-contact. Additionally, various ambient noise impedes the public applications of voice user interfaces due to the requirement of audio-only speech recognition methods for a high signal-to-noise ratio. In this paper, we present Wavoice, the first noise-resistant multi-modal speech recognition system that fuses two distinct voice sensing modalities, i.e., millimeter-wave (mmWave) signals and audio signals from a microphone, together. One key contribution is that we model the inherent correlation between mmWave and audio signals. Based on it, Wavoice facilitates the real-time noise-resistant voice activity detection and user targeting from multiple speakers. Furthermore, we elaborate on two novel modules into the neural attention mechanism for multi-modal signals fusion, and result in accurate speech recognition. Extensive experiments verify Wavoice's effectiveness under various conditions with the character recognition error rate below 1% in a range of 7 meters. Wavoice outperforms existing audio-only speech recognition methods with lower character error rate and word error rate. The evaluation in complex scenes validates the robustness of Wavoice.

Playing is an essential way for preschoolers to learn. There are three types of games for preschoolers: functional play, constructive play, and symbolic play. However, existing works/games can only work for one specific type of children's play. Therefore, we designed and implemented an extensible children's game framework based on Augmented Reality building blocks. We implement an AR prototype from scratch that achieves up to 8 blocks with 81% detection rate and overhead of 21ms (46 FPS) on average.

Virtual Reality (VR) training is an emerging method, which is widely deployed in more and more applications. Compared with traditional physical training and video games-based training, VR training can not only provide a sense of realism and immersion similar to physical training but can also train at any time and place, saving time and money. However, due to some constraints like lacking reflections of the ambient environment, the realism and immersion of VR training are insufficient. Therefore, in this paper, we propose enhanced VR training which senses the ambient environment and reflects them as dynamic unexpected training tasks to solve the above problems.

To address the increasing challenges of counterfeit detection and IP protection for 3D printing, we propose that every 3D printer holds unique fingerprinting features characterized by the thermodynamic properties of the extruder hot-end and can be used as a new way of 3D watermarking. We prove that these physical fingerprints resulting from manufacturing imperfections and system variations exhibit distinct heating responses, namely “ThermoTag,” which can be represented as the distinguishable thermodynamic processes and, ultimately, the temperature readings during the preheating process. Experimental results show that, by only changing the hot-ends of the same model on the same 3D printer, we can achieve about 92% identification accuracy amongst 45 hot-ends. The permanence and robustness of ThermoTag for the same hot-end were examined, throughout a period of one month with hundreds of trials under different environmental temperature settings. Leveraging the hidden ThermoTag, an example of watermarking scheme in 3D printing is presented and evaluated.

Falls are leading causes of nonfatal injuries in workplaces which lead to substantial injury and economic consequences. To help avoid fall injuries, safety managers usually need to inspect working areas routinely. However, it is difficult for a limited number of safety managers to inspect fall hazards instantly especially in large workplaces. To address this problem, a novel fall hazard identification method is proposed in this paper which makes it possible for all workers to report the potential hazards automatically. This method is based on the fact that people use different gaits to get across different floor surfaces. Through analyzing gait patterns, potential fall hazards could be identified automatically. In this research, Smart Insole, an insole shaped wearable system for gait analysis, was applied to measure gait patterns for fall hazard identification. Slips and trips are the focus of this study since they are two main causes of falls in workplaces. Five effective gait features were extracted to train a Support Vector Machine (SVM) model for recognizing slip hazard, trip hazard, and safe floor surfaces. Experiment results showed that fall hazards could be recognized with high accuracy (98.1%).

The multi-interface networks are efficient infrastructures to deploy distributed Deep Learning (DL) tasks as the model gradients generated by each worker can be exchanged to others via different links in parallel. Although this decentralized parameter synchronization mechanism can reduce the time of gradient exchange, building a high-performance distributed DL architecture still requires the balance of communication efficiency and computational utilization, i.e., addressing the issues of traffic burst, data consistency, and programming convenience. To achieve this goal, we intend to asynchronously exchange gradient pieces without the central control in multi-interface networks. We propose the Piece-level Gradient Exchange and Multi-interface Collective Communication to handle parameter synchronization and traffic transmission, respectively. Specifically, we design the gradient sketch approach based on 8-bit uniform quantization to compress gradient tensors and introduce the colayerabstraction to better handle gradient partition, exchange and pipelining. Also, we provide general programming interfaces to capture the synchronization semantics and build the Gradient Exchange Index (GEI) data structures to make our approach online applicable. We implement our algorithms into a prototype system called Canary by using PyTorch-1.4.0. Experiments conducted in Alibaba Cloud demonstrate that Canary reduces 56.28 percent traffic on average and completes the training by up to 1.61x, 2.28x, and 2.84x faster than BML, Ako on PyTorch, and PS on TensorFlow, respectively.

Deep Neural Networks (DNNs) are now presenting human-level performance on many real-world applications, and DNN-based intelligent services are becoming more and more popular across all aspects of our lives. Unfortunately, the ever-increasing DNN service implies a dangerous feature which has not yet been well studied–allowing the marriage of existing malware and DNN model for any pre-defined malicious purpose. In this paper, we comprehensively investigate how to turn DNN into a new breed evasive self-contained stegomalware, namely StegoNet, using model parameter as a novel payload injection channel, with no service quality degradation (i.e. accuracy) and the triggering event connected to the physical world by specified DNN inputs. A series of payload injection techniques which take advantage of a variety of unique neural network natures like complex structure, high error resilience capability and huge parameter size, are developed for both uncompressed models (with model redundancy) and deeply compressed models tailored for resource-limited devices (no model redundancy), including LSB substitution, resilience training, value mapping, and sign-mapping. We also proposed a set of triggering techniques like logits trigger, rank trigger and fine-tuned rank trigger to trigger StegoNet by specific physical events under realistic environment variations. We implement the StegoNet prototype on Nvidia Jetson TX2 testbed. Extensive experimental results and discussions on the evasiveness, integrity of proposed payload injection techniques, and the reliability and sensitivity of the triggering techniques, well demonstrate the feasibility and practicality of StegoNet.

Increasing malicious users have sought practices to leverage 3D printing technology to produce unlawful tools in criminal activities. It is of vital importance to enable 3D printers to identify the objects to be printed and terminate at early stage if illegal objects are identified. Deep learning yields significant rises in performance in the object recognition tasks. However, the lack of large-scale databases in 3D printing domain stalls the advancement of automatic illegal weapon recognition. This paper presents a new 3D printing image database, namely C3PO, which compromises two subsets for the different system working scenarios. We extract images from the numerical control programming code files of 22 3D models, and then categorize the images into 10 distinct labels. These two sets are designed for identifying: (i). printing knowledge source (G-code) at beginning of manufacturing, (ii). printing procedure during manufacturing. Importantly, we demonstrate that the weapons can be recognized in either scenario using deep learning based approaches using our proposed database. The quantitative results are promising, and the future exploration of the database and the crime prevention in 3D printing are demanding tasks.

Smartphones have recently become a popular platform for deploying the computation-intensive virtual reality (VR) applications, such as immersive video streaming (a.k.a., 360-degree video streaming). One specific challenge involving the smartphone-based head mounted display (HMD) is to reduce the potentially huge power consumption caused by the immersive video. To address this challenge, we first conduct an empirical power measurement study on a typical smartphone immersive streaming system, which identifies the major power consumption sources. Then, we develop QuRate, a quality-aware and user-centric frame rate adaptation mechanism to tackle the power consumption issue in immersive video streaming. QuRate optimizes the immersive video power consumption by modeling the correlation between the perceivable video quality and the user behavior. Specifically, QuRate builds on top of the user's reduced level of concentration on the video frames during view switching and dynamically adjusts the frame rate without impacting the perceivable video quality. We evaluate QuRate with a comprehensive set of experiments involving 5 smartphones, 21 users, and 6 immersive videos using empirical user head movement traces. Our experimental results demonstrate that QuRate is capable of extending the smartphone battery life by up to 1.24X while maintaining the perceivable video quality during immersive video streaming. Also, we conduct an Institutional Review Board (IRB)-approved subjective user study to further validate the minimum video quality impact caused by QuRate.

Temperature sensor is one of the most widespread technologies in the IoT era. Wireless temperature monitoring systems are convenient to deploy and can drive mass applications in the fields of smart home, transportation and logistics. Currently, wireless temperature monitoring products are based on microelectronic and semiconductor components, which are not cost-effective (e.g., a few dollars) and more importantly, generate electronic wastes. In this work, we present ThermoWave, a new paradigm of wireless temperature monitoring that is ecological, battery-less, and ultra-low cost. Specifically, ThermoWave is on the basis of the thermal scattering effect on millimeter-wave (mmWave) signals. Specifically, cholesteryl materials align their molecular patterns at different environmental temperatures, and this temperature-induced pattern change will be modulated and sensed by the scattered mmWave signals. There are three functional modules in the ThermoWave system. The ThermoTag is a cholesteryl material inked film or paper tag that can be conveniently attached to the object of interest to monitor temperature changes. Each ThermoTag costs less than 0.01 dollars. The temperature modulated mmWave scattering will be received by a mmWave-radar based ThermoScanner and demodulated by a software-based temperature decoder ThermoSense, which includes a model-based method (i.e., ThermoDot) for point temperature estimation and a data-driven method (i.e., ThermoNet) for thermal imaging. We prototype and evaluate the ThermoWave system performance in both controlled and real-world setups. Experimental results show that the ThermoWave achieves the precision of ±1.0°F in the range of 30°F to 120°F in a controlled setup. We also investigate the performance in real-world applications, and the ThermoWave can reach the ±3.0°F precision in the temperature estimation. We also test and discuss sustainability, durability, robustness, and cost-effectiveness of the ThermoWave in both design and experiments.

Internet of Things (IoT) devices are hindered from communicating with their neighbors by incompatible protocols or electromagnetic interference. Existing solutions adopting physical covert channels have limitations in receiver distinction, additional hardware, conditional placement, or physical contact. Our system, Deaf-Aid, utilizes the stealthy speaker-to-gyroscope channel to build robust protocol-independent communication with automatic receiver identification. Deaf-Aid exploits ultrasonic signals at a frequency corresponding to the target receiver, forcing the gyroscope inside to resonate, so as to convey information. We probe the relationship among axes in a gyroscope to surmount frequency offset ingeniously and support multi-channel communication. Meanwhile, Deaf-Aid identifies the receivers automatically via device fingerprints constituted by the diversity of resonant frequency ranges. Furthermore, we entitle Deaf-Aid the capability of mobile communication which is an essential demand for IoT devices. We address the challenge of accurate signals recovery from motion interference. Extensive evaluations demonstrate that Deaf-Aid yields 47bps with BER lower than 1% under motion interference. To our best knowledge, Deaf-Aid is the first work to enable stealthy mobile IoT communication on the basis of inertial motion sensors.

Drug effectiveness management is a complicated and challenging task in chronic diseases, like Parkinson's Disease (PD). Drug effectiveness control is not only linked to personal out-of-pocket cost but also affecting the quality of life among patients with chronic symptoms. In the current practice, although that health and medical professionals still play a key role in the personalized treatment plan, the critical decision on drug selection falls upon the individual report when patients call in or visit the clinics. Unfortunately, most of the patients with chronic diseases either fail to report their day-to-day symptoms or have a limited access to medical resources due to economic constraints. In this paper, we present PDLens, a first smartphone-based system to detect drug effectiveness among Parkinson's in daily life. Specifically, PDLens can extract digital behavioral markers related to PD drug responses from everyday activities, including phone calls, standing, and walking. PDLens models the PD symptom severity on drug treatment and detects the change of severity scores before and after drug intake. A ranking-based multi-view deep neural network is developed to decide the drug effectiveness upon the symptom severity changes. To validate the performance of PDLens, we conduct a pilot study with 81 PD patients and monitor their smartphone activities and severity changes over 33693 drug intake events across six (6) months. Compared with the standard clinical drug effectiveness test developed by Motor Disorder Society, results reveal that PDLens is a promising tool to facilitate drug effectiveness detection among PD patients in their daily lives.

The advent of smart devices has caused unprecedented security and privacy concerns to its users. Although the fingerprint technology is a go-to biometric solution in high-impact applications (e.g., smart-phone security, monetary transactions and international-border verification), the existing fingerprint scanners are vulnerable to spoofing attacks via fake-finger and cannot be employed across smart devices (e.g., wearables) due to hardware constraints. We propose SonicPrint that extends fingerprint identification beyond smartphones to any smart device without the need for traditional fingerprint scanners. SonicPrint builds on the fingerprint-induced sonic effect (FiSe) caused by a user swiping his fingertip on smart devices and the resulting property, i.e., different users' fingerprint would result in distinct FiSe. As the first exploratory study, extensive experiments verify the above property with 31 participants over four different swipe actions on five different types of smart devices with even partial fingerprints. SonicPrint achieves up to a 98% identification accuracy on smartphone and an equal-error-rate (EER) less than 3% for smartwatch and headphones. We also examine and demonstrate the resilience of SonicPrint against fingerprint phantoms and replay attacks. A key advantage of SonicPrint is that it leverages the already existing microphones in smart devices, requiring no hardware modifications. Compared with other biometrics including physiological patterns and passive sensing, SonicPrint is a low-cost, privacy-oriented and secure approach to identify users across smart devices of unique form-factors.

Approximately 7 million survivors of stroke reside in the United States. Over half of these individuals will have residual deficits, making stroke one of the leading causes of disability. Long-term rehabilitation opportunities are critical for millions of individuals with chronic upper limb motor deicits due to stroke. Traditional in-home rehabilitation is reported to be dull, boring, and un-engaging. Moreover, existing rehabilitation technologies are not user-friendly and cannot be adaptable to different and ever-changing demands from individual stroke survivors. In this work, we present RehabPhone, a highly-usable software-defined stroke rehabilitation paradigm using the smartphone and 3D printing technologies. This software-definition has twofold. First, RehabPhone leverages the cost-effective 3D printing technology to augment ordinal smartphones into customized rehabilitation tools. The size, weight, and shape of rehabilitation tools are software-defined according to individual rehabilitation needs and goals. Second, RehabPhone integrates 13 functional rehabilitation activities co-designed with stroke professionals into a smartphone APP. The software utilizes built-in smartphone sensors to analyzes rehabilitation activities and provides real-time feedback to coach and engage stroke users. We perform the in-lab usability optimization with the RehabPhone prototype with involving 16 healthy adults and 4 stroke survivors. After that, we conduct a 6-week unattended intervention study in 12 homes of stroke residence. In the course of the clinical study, over 32,000 samples of physical rehabilitation activities are collected and evaluated. Results indicate that stroke users with RehabPhone demonstrate a high adherence and clinical efficacy in a self-managed home-based rehabilitation course. To the best of our knowledge, this is the first exploratory clinical study using mobile health technologies in real-world stroke rehabilitation.

Core body temperature is an important indicator of medical treatment. Sudden changes in core body temperature can be a precursor to neurodegenerative diseases such as Parkinson's disease. These diseases have the potential to strike at any time, therefore, long-term monitoring of core body temperature and alerting to sudden changes in temperature become important. In this paper, we designed an in-ear thermometer to monitor the core body temperature with the help of smartphone.

With the continuing growth of voice-controlled devices, voice metrics have been widely used for user identification. However, voice biometrics is vulnerable to replay attacks and ambient noise. We identify that the fundamental vulnerability in voice biometrics is rooted in its indirect sensing modality (e.g., microphone). In this paper, we present VocalPrint, a resilient mmWave interrogation system which directly captures and analyzes the vocal vibrations for user authentication. Specifically, VocalPrint exploits the unique disturbance of the skin-reflect radio frequency (RF) signals around the near-throat region of the user, caused by the vocal vibrations during communication. The complex ambient noise is isolated from the RF signal using a novel resilience-aware clutter suppression approach for preserving fine-grained vocal biometric properties. Afterward, we extract the text-independent vocal tract and vocal source features and input them to an ensemble classifier for user authentication. VocalPrint is practical as it leverages a low-cost, portable, and energy-efficient hardware allowing effortless transition to a smartphone while having sufficient usability as typical voice authentication systems due to its non-contact nature. Our experimental results from 41 participants with different interrogation distances, orientations, and body motions show that VocalPrint can achieve over 96% authentication accuracy even under unfavorable conditions. We demonstrate the resilience of our system against complex noise interference and spoof attacks of various threat levels.

Digital screens, such as liquid crystal displays (LCDs), are vulnerable to attacks (e.g., "shoulder surfing") that can bypass security protection services (e.g., firewall) to steal confidential information from intended victims. The conventional practice to mitigate these threats is isolation. An isolated zone, without accessibility, proximity, and line-of-sight, seems to bring personal devices to a truly secure place.In this paper, we revisit this historical topic and re-examine the security risk of screen attacks in an isolation scenario mentioned above. Specifically, we identify and validate a new and practical side-channel attack for screen content via liquid crystal nematic state estimation using a low-cost radio-frequency sensor. By leveraging the relationship between the screen content and the states of liquid crystal arrays in displays, we develop WaveSpy, an end-to-end portable through-wall screen attack system. WaveSpy comprises a low-cost, energy-efficient and light-weight millimeter-wave (mmWave) probe which can remotely collect the liquid crystal state response to a set of mmWave stimuli and facilitate screen content inference, even when the victim’s screen is placed in an isolated zone. We intensively evaluate the performance and practicality of WaveSpy in screen attacks, including over 100 different types of content on 30 digital screens of modern electronic devices. WaveSpy achieves an accuracy of 99% in screen content type recognition and a success rate of 87.77% in Top-3 sensitive information retrieval under real-world scenarios, respectively. Furthermore, we discuss several potential defense mechanisms to mitigate screen eavesdropping similar to WaveSpy.

Abstract Classification of electrocardiogram (ECG) signals is obligatory for the automatic diagnosis of cardiovascular disease. With the recent advancement of low‐cost wearable ECG device, it becomes more feasible to utilize ECG for cardiac arrhythmia classification in daily life. In this paper, we propose a lightweight approach to classify five types of cardiac arrhythmia, namely, normal beat (N), atrial premature contraction (A), premature ventricular contraction (V), left bundle branch block beat (L), and right bundle branch block beat (R). The combined method of frequency analysis and Shannon entropy is applied to extract appropriate statistical features. Information gain criterion is employed to select features that the results show that 10 highly effective features can obtain performance measures comparable to those obtained by using the complete features. The selected features are then fed to the input of Random Forest, K‐Nearest Neighbour, and J48 for classification. To evaluate classification performance, tenfold cross validation is used to verify the effectiveness of our method. Experimental results show that Random Forest classifier demonstrates significant performance with the highest sensitivity of 98.1%, the specificity of 99.5%, the precision of 98.1%, and the accuracy of 98.08%, outperforming other representative approaches for automated cardiac arrhythmia classification.

With the development of the Internet of Things (IoT), wearable technologies have been proposed to measure gait parameters in everyday life. However, since both diseases and activities could influence gait patterns, clinicians cannot use the measured gait parameters for clinical applications without knowing the corresponding activities. To address this problem, a novel gait analysis method—“gait analysis in terms of activities of daily living (ADLs)”—was proposed based on a wearable Smart Insole system. Twenty six gait parameters were extracted to realize a systematic gait analysis. Novel activity recognition algorithms based on characteristics of human gait were proposed to recognize ADLs, including “sitting,” “standing,” “walking,” “running,” “ascend stairs,” and “descend stairs” with high accuracy and low computation load. To evaluate the performance of “gait analysis in terms of ADLs,” an experiment consisting of a sequence of different ADLs was designed to simulate the scenario of everyday life. In the result, gait parameters measured during different activities were automatically highlighted with different colors, which made it easy to see whether the gait pattern change was caused by activities or diseases. Besides, a refined gait analysis could be realized by individually extracting and analyzing the gait parameters of a specific activity. The results indicate that “gait analysis in terms of ADLs” is a feasible method to reach the aim of bringing gait lab to everyday life.

Biometric authentication offers advantages over current security practices. Unlike keys and tokens, biometrics are never lost or stolen. Unlike passwords, biometrics cannot be forgotten. However, existing biometric systems are with controversy: once divulged, they are compromised forever. To this end, this paper explores a truly cancelable brain-based biometric system for the first time. Specifically, we present a new psychophysiological protocol via non-volitional brain response for trustworthy user authentication, with an application example of smart headwear. More specifically, we address the following research challenges in a theoretical and experimental combined manner: (1) how to generate reliable brain responses with sophisticated visual stimuli; (2) how to acquire effective brain response and analyze unique features in them for authentication; and (3) how to reset and change brain biometrics when the current biometric credential is divulged. To evaluate the performance of the proposed system, we conducted a pilot study and achieved an f-score accuracy of 95.46 percent and equal error rate (EER) of 2.503 percent, thereby demonstrating the potential feasibility of neurofeedback based biometrics for smart headwear applications. Further, the cancelability study proves the effectiveness of the reset brain password. To the best of our knowledge, it is the first in-depth research study on truly cancelable brain biometrics.

Model compression is an important technique to facilitate efficient embedded and hardware implementations of deep neural networks (DNNs), a number of prior works are dedicated to model compression techniques. The target is to simultaneously reduce the model storage size and accelerate the computation, with minor effect on accuracy. Two important categories of DNN model compression techniques are weight pruning and weight quantization. The former leverages the redundancy in the number of weights, whereas the latter leverages the redundancy in bit representation of weights. These two sources of redundancy can be combined, thereby leading to a higher degree of DNN model compression. However, a systematic framework of joint weight pruning and quantization of DNNs is lacking, thereby limiting the available model compression ratio. Moreover, the computation reduction, energy efficiency improvement, and hardware performance overhead need to be accounted besides simply model size reduction, and the hardware performance overhead resulted from weight pruning method needs to be taken into consideration. To address these limitations, we present ADMM-NN, the first algorithm-hardware co-optimization framework of DNNs using Alternating Direction Method of Multipliers (ADMM), a powerful technique to solve non-convex optimization problems with possibly combinatorial constraints. The first part of ADMM-NN is a systematic, joint framework of DNN weight pruning and quantization using ADMM. It can be understood as a smart regularization technique with regularization target dynamically updated in each ADMM iteration, thereby resulting in higher performance in model compression than the state-of-the-art. The second part is hardware-aware DNN optimizations to facilitate hardware-level implementations. We perform ADMM-based weight pruning and quantization considering (i) the computation reduction and energy efficiency improvement, and (ii) the hardware performance overhead due to irregular sparsity. The first requirement prioritizes the convolutional layer compression over fully-connected layers, while the latter requires a concept of the break-even pruning ratio, defined as the minimum pruning ratio of a specific layer that results in no hardware performance degradation. Without accuracy loss, ADMM-NN achieves 85× and 24× pruning on LeNet-5 and AlexNet models, respectively, --- significantly higher than the state-of-the-art. The improvements become more significant when focusing on computation reduction. Combining weight pruning and quantization, we achieve 1,910× and 231× reductions in overall model size on these two benchmarks, when focusing on data storage. Highly promising results are also observed on other representative DNNs such as VGGNet and ResNet-50. We release codes and models at https://github.com/yeshaokai/admm-nn.

The ill-fitting shoes can cause many health implications, ranging from feet ache to feet sore, from back injury to back pain and so on. Due to the manufacturing variance and design consideration, shoes from different brands will vary in size and shape. Good-fit insoles and shoes are necessary to prevent users from ill-fitting problems. Hence, we propose a novel technique that can be used to calculate parameters of the foot from smartphone RGB camera inputs. Typically, we focus on estimating six main parameters of the foot, including foot length, foot width, foot back height, toe height, heel circumference and foot circumference. These six parameters will significantly help users to find the right shoes or order a tailor-made insoles that fits their foot condition so that they can sustain their foot health and avoid further damage. Therefore, we applied different mathematical formulas accordingly and designed algorithms to estimate foot parameters.

Deep learning solutions are being increasingly deployed in mobile applications, at least for the inference phase. Due to the large model size and computational requirements, model compression for deep neural networks (DNNs) becomes necessary, especially considering the real-time requirement in embedded systems. In this paper, we extend the prior work on systematic DNN weight pruning using ADMM (Alternating Direction Method of Multipliers). We integrate ADMM regularization with masked mapping/retraining, thereby guaranteeing solution feasibility and providing high solution quality. Besides superior performance on representative DNN benchmarks (e.g., AlexNet, ResNet), we focus on two new applications facial emotion detection and eye tracking, and develop a top-down framework of DNN training, model compression, and acceleration in mobile devices. Experimental results show that with negligible accuracy degradation, the proposed method can achieve significant storage/memory reduction and speedup in mobile devices.

Recurrent Neural Networks (RNNs) are becoming increasingly important for time series-related applications which require efficient and real-time implementations. The two major types are Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) networks. It is a challenging task to have real-time, efficient, and accurate hardware RNN implementations because of the high sensitivity to imprecision accumulation and the requirement of special activation function implementations. Recently two works have focused on FPGA implementation of inference phase of LSTM RNNs with model compression. First, ESE uses a weight pruning based compressed RNN model but suffers from irregular network structure after pruning. The second work C-LSTM mitigates the irregular network limitation by incorporating block-circulant matrices for weight matrix representation in RNNs, thereby achieving simultaneous model compression and acceleration. A key limitation of the prior works is the lack of a systematic design optimization framework of RNN model and hardware implementations, especially when the block size (or compression ratio) should be jointly optimized with RNN type, layer size, etc. In this paper, we adopt the block-circulant matrixbased framework, and present the Efficient RNN (E-RNN) framework for FPGA implementations of the Automatic Speech Recognition (ASR) application. The overall goal is to improve performance/energy efficiency under accuracy requirement. We use the alternating direction method of multipliers (ADMM) technique for more accurate block-circulant training, and present two design explorations providing guidance on block size and reducing RNN training trials. Based on the two observations, we decompose E-RNN in two phases: Phase I on determining RNN model to reduce computation and storage subject to accuracy requirement, and Phase II on hardware implementations given RNN model, including processing element design/optimization, quantization, activation implementation, etc. 1 Experimental results on actual FPGA deployments show that E-RNN achieves a maximum energy efficiency improvement of 37.4× compared with ESE, and more than 2× compared with C-LSTM, under the same accuracy.

Parkinson's disease (PD) is a chronic neurodegenerative disorder resulting from the progressive loss of dopaminergic nerve cells. People with PD usually demonstrate deficits in performing basic daily activities, and the relevant annual social cost can reach about $25 billion in the United States. Early detection of PD plays an important role in symptom relief and improvement in the performance of activities in daily life (ADL), which eventually reduces societal and economic burden. However, conventional PD detection methods are inconvenient in daily life (e.g., requiring users to wear sensors). To overcome this challenge, we propose and identify the non-speech body sounds as the new PD biomarker, and utilize the data in smartphone usage to realize the passive PD detection in daily life without interrupting the user. Specifically, we present PDVocal, an end-to-end smartphone-based privacy-preserving system towards early PD detection. PDVocal can passively recognize the PD digital biomarkers in the voice data during daily phone conversation. At the user end, PDVocal filters the audio stream and only extracts the non-speech body sounds (e.g., breathing, clearing throat and swallowing) which contain no privacy-sensitive content. At the cloud end, PDVocal analyzes the body sounds of interest and assesses the health condition using a customized residual network. For the sake of reliability in real-world PD detection, we investigate the method of the performance optimizer including an opportunistic learning knob and a long-term tracking protocol. We evaluate our proposed PDVocal on a collected dataset from 890 participants and real-life conversations from publicly available data sources. Results indicate that non-speech body sounds are a promising digital biomarker for privacy-preserving PD detection in daily life.

In recent years, significant research efforts have been spent towards building intelligent and user-friendly IoT systems to enable a new generation of applications capable of performing complex sensing and recognition tasks. In many of such applications, there are usually multiple different sensors monitoring the same object. Each of these sensors can be regarded as an information source and provides us a unique "view" of the observed object. Intuitively, if we can combine the complementary information carried by multiple sensors, we will be able to improve the sensing performance. Towards this end, we propose DeepFusion, a unified multi-sensor deep learning framework, to learn informative representations of heterogeneous sensory data. DeepFusion can combine different sensors' information weighted by the quality of their data and incorporate cross-sensor correlations, and thus can benefit a wide spectrum of IoT applications. To evaluate the proposed DeepFusion model, we set up two real-world human activity recognition testbeds using commercialized wearable and wireless sensing devices. Experiment results show that DeepFusion can outperform the state-of-the-art human activity recognition methods.

Digital devices have become a necessity in our daily life, with digital screens acting as a gateway to access a plethora of information present in the underlying device. However, these devices emit visible light through screens where long-term use can lead to significant screen exposure, further influencing users' health. Conventional methods on screen exposure detection (\textite.g., photo logger) are usually privacy-invasive and expensive, further, require ideal light conditions, which are unattainable in real practice. Considering the light intensity and spectrum vary among different light sources, an effective screen spectrum estimation can provide vital information about screen exposure. To this end, we first investigate the characteristics of the junction between p-type and n-type semiconductor (i.e., PN junction) to sense the spectrum under various conditions. Empirically, we design and implement, \textsfSpecEye, an end-to-end, low cost, wearable, and privacy-preserving screen exposure detection system with a mobile application. For validating the performance of our system, we conduct comprehensive experiments with $54$ commodity digital screens, at $43$ distinct locations, with results showing a base accuracy of $99$%, and an equal error rate (EER) approaching $0.80$% under the controlled lab setup. Moreover, we assess the reliability, robustness, and performance variation of \textsfSpecEye under various real-world circumstances to observe a stable accuracy of $95$%. Our real-world study indicates \textsfSpecEye is a promising system for screen exposure detection in everyday life.

Voice-user interface (VUI) has become an integral component in modern personal devices (\textite.g., smartphones, voice assistant) by fundamentally evolving the information sharing between the user and device. Acoustic sensing for VUI is designed to sense all acoustic objects; however, the existing VUI mechanism can only offer low-quality speech sensing. This is due to the audible and inaudible interference from complex ambient noise that limits the performance of VUI by causing denial-of-service (DoS) of user requests. Therefore, it is of paramount importance to enable noise-resistant speech sensing in VUI for executing critical tasks with superior efficiency and precision in robust environments. To this end, we investigate the feasibility of employing radio-frequency signals, such as millimeter wave (mmWave) for sensing the noise-resistant voice of an individual. We first perform an in-depth study behind the rationale of voice generation and resulting vocal vibrations. From the obtained insights, we presentWaveEar, an end-to-end noise-resistant speech sensing system.WaveEar comprises a low-cost mmWave probe to localize the position of the speaker among multiple people and direct the mmWave signals towards the near-throat region of the speaker for sensing his/her vocal vibrations. The received signal, containing the speech information, is fed to our novel deep neural network for recovering the voice through exhaustive extraction. Our experimental evaluation under real-world scenarios with 21 participants shows the effectiveness ofWaveEar to precisely infer the noise-resistant voice and enable a pervasive VUI in modern electronic devices.

Due to the limited battery capacity of mobile devices, various CPU power governors and dynamic frequency adjustment schemes have been proposed to reduce CPU energy consumption. However, most such schemes are app-oblivious, ignoring an important fact that real-world applications often exhibit multiple execution phases that perform different functionality and may request different amounts of hardware resources. Having a unified app-level frequency setting for different phases of an application may not be energy efficient enough and may even violate the desirable latency performance required by certain phases. Motivated by this observation, in this paper, we present PIFA, which is an intelligent Phase Identification and Frequency Adjustment framework for energy-efficient and time-sensitive mobile computing. PIFA addresses two major challenges of fully automatically identifying different execution phases of an application and efficiently integrating the phase identification results for runtime frequency adjustment. We have fully implemented PIFA on the Android platform. An extensive set of experiments using real-world Android applications from multiple app categories demonstrate that PIFA achieves closely better performance than the desired latency requirement specified for each phase, while dramatically reducing energy consumption (e.g., >30% energy reduction for most apps) and incurring rather small runtime overhead (e.g., <;5% overhead for most apps).

Hidden electronics possess the risk of both security threat and privacy intrusion. We present a wireless hidden electronic recognition system, through electronic components unique mmWave nonlinear responses to identify the threats. We then evaluate E-Eye's performance and robustness with a controlled experiment and a field study using iconic devices and score the system with metrics. Results prove that E-Eye is an accurate and robust hidden electronic recognition system.

Effective home rehabilitation is important for recovery of hand grip ability in post-stroke individuals. This paper presents BIGHand, a bilateral, integrated, and gamified handgrip stroke rehabilitation system for independent at-home exercise. BIGHand consists of affordable sensor-integrated hardware (Vernier hand dynamometers, Arduino Uno, interface shield) used to obtain real-time grip force data, and a set of exergames designed as parts of an interactive structural rehabilitation program. This program pairs targeted difficulty progression with user-ability scaled controls to create an adaptive, challenging, and enticing rehabilitation environment. This training prepares users for the many activities of daily living (ADLs) by targeting strength, bilateral coordination, hand-eye coordination, speed, endurance, precision, and dynamic grip force adjustment. Multiple measures are taken to engage, motivate, and guide users through the at-home rehabilitation process, including "smart" post-game feedback and in-game goals. A demo video is available at https://youtu.be/zrLVkZZ4Ukc.

Effective at-home rehabilitation of both upper and lower extremities is important for regaining proficiency in activities of daily living (ADLs) post-stroke. We introduce ARMove, a smartphone augmented reality (AR) exergaming system for upper and lower extremities stroke rehabilitation. The AR technology facilitates exergaming that utilizes full range of motion in real-world spatial environments, while creating interesting graphics to engage users in gamified environments. ARMove's novelty comes from its multifaceted rehabilitation of both upper and lower extremities. Furthermore, ARMove provides simultaneous training of fine and gross movements; it also considers bilateral training, preparing users for ADLs such as using computers or playing sports. Additionally, our utilization of smartphone embedded vision sensors and mobile computing give our system scalability, with potential for ubiquitous deployment.

Inventory management is pivotal in the supply chain to supervise the non-capitalized products and stock items. Item counting, indexing and identification are the major jobs of inventory management. Currently, the most adopted inventory technologies in product counting/identification are using either the laser-scannable barcode or the radio-frequency identification (RFID). However, the laser-scannable barcode is entangled by an alignment issue (i.e., the laser reader must align with one barcode in line-of-sight), and the RFID is economically and environmentally unfriendly (i.e., high-cost and not naturally disposable). To this end, we propose FerroTag which is a paper-based mmWave-scannable tagging infrastructure for the next generation inventory management system, featuring ultra-low cost, environment-friendly, battery-free and in-situ (i.e., multiple tags can be simultaneously processed outside the line-of-sight). FerroTag is developed on top of the FerroRF effects. Specifically, the magnetic nanoparticles within the ferrofluidic ink reply to probing mmWave with classifiable features (i.e., the FerroRF response). By designating the ink pattern and hence the location of particles, the related FerroRF response can be modified. Thus, a specifically designated ferrofluidic ink printed pattern, which is associated with a unique FerroRF response, is a remotely retrievable (a.k.a., mmWave-scannable) identity. Furthermore, we augment FerroTag by designing a high capacity pattern system and a fine-grained identification protocol such that the capacity and robustness of FerroTag can be systematically improved in mass product management in inventory. Last but not least, we evaluate the performance of FerroTag with 201 different tag design patterns. Results show that FerroTag can identify tags with an accuracy of more than 99% in a controlled lab setup. Moreover, we examine the reliability, robustness and performance of FerroTag under various real-world circumstances, where FerroTag maintains the accuracy over 97%. Therefore, FerroTag is a promising tagging infrastructure for the applications in inventory management systems.

Continuous authentication is superior to conventional one-pass authentication by maintaining the security level of a system throughout the entire login session. Leveraging the unique geometric and non-volitional credentials in the cardiac motion, we present a trustworthy, continuous, and non-contact user authentication system. Based on a pilot study with 78 subjects, we evaluate Cardiac Scan in terms of accuracy, authentication time, permanence, and vulnerability. The results show that Cardiac Scan is a robust and usable continuous authentication system.

With the development of Internet of Things (IoT), indoor localization has been a research focus in recent years. For inertial measurement unit (IMU)-based indoor localization method, zero velocity update (ZUPT) uses the known velocity at stationary epoch as a benchmark to calibrate the velocity drift. However, stationary epoch only takes up 24% of a whole gait cycle time, and the velocity drift at the remaining 76% time is usually estimated according to an assumption that velocity has a linear drift over time, which would introduce errors. In this paper, a two-step velocity calibration method was proposed based on human gait characteristics with Smart Insole: known velocity update (KUPT) and double-foot position calibration (DFPC). KUPT could measure the velocity from heel-strike to toe-off based on the recorded real-time foot angle and the shoe dimensions, which increases the time period when the velocity could be measured from 24% to 62% of a whole gait cycle time. DFPC method could fuse the position information of both feet based on the symmetrical characteristic of human gait to further increase the reliability of the localization results. The statistical result of a 20 times 20-m walking experiment showed that KUPT method was more accurate and reliable than ZUPT method for both feet, and DFPC method could further improve the result of KUPT method. Another experiment about walking in an indoor environment for 91 m showed that the proposed KUPT+DFPC method had an error of about 0.78 m which is acceptable for most IoT applications.

With the growth of the elder population, fall risk evaluation is crucial to prevent elders from serious injuries, as well as reduce related financial burdens. A balance assessment, timed up and go (TUG), has been widely used to estimate fall risk. The standardized TUG focuses on flat ground walking with no environmental variance. Therefore, it falls short of assessing an individual's gait adaptability. Being able to adjust steps in response to environmental changes, for example, needing to navigate around or over a child's toy left on the sidewalk, is essential to avoid fall risk and fundamental to community ambulation. To this end, we propose four environment-adapting TUGs designed to assess one's ability to adapt gait in complex environments and a compatible system named Smart Insole TUG (SITUG), which provides real-time, feature-rich, and ease-of-operation TUG analysis. Based on experimental results, SITUG is capable of extracting gait related spatial-temporal features with all mean accuracies over 92%. Besides, the system achieves a mean accuracy of 92.23% in segmenting five TUG phases.

The rapid development of Internet-of-Things is yielding a huge volume of time series data, the real-time mining of which becomes a major load for data centers. The computation bottleneck in time series data mining is distance function, which is the fundamental element of many high data mining tasks. Recently various software optimization and hardware acceleration techniques have been proposed to tackle the challenge. However, each of these techniques is only designed or optimized for a specific distance function. To address this problem, in this paper we propose MDA, a high-throughput reconfigurable memristor-based distance accelerator for real-time and energy-efficient data mining with time series in data centers. Common circuit structure is extracted for efficiency, and the circuit can be configured to any specific distance functions. Particularly, we adopt the emerging device memristor for the design of MDA. Comprehensive experiments are presented with public available datasets to evaluate the performance of the proposed MDA. Experimental results show that compared with existing works, MDA has achieved a speedup of 3.5×-376× on performance and an improvement of 1-3 orders of magnitude on energy efficiency with little accuracy loss.

While the intersection of blockchain and Industrial Internet of Things (IIoT) has received considerable research interest lately, the conflict between the high resource requirements of blockchain and the generally inadequate performance of IIoT devices has not been well tackled. On one hand, due to the introductions of mathematical concepts, including Public Key Infrastructure, Merkle Hash Tree, and Proof of Work (PoW), deploying blockchain demands huge computing power. On the other hand, full nodes should synchronize massive block data and deal with numerous transactions in peer-to-peer network, whose occupation of storage capacity and bandwidth makes IIoT devices difficult to afford. In this paper, we propose a lightweight blockchain system called LightChain, which is resource-efficient and suitable for power-constrained IIoT scenarios. Specifically, we present a green consensus mechanism named Synergistic Multiple Proof for stimulating the cooperation of IIoT devices, and a lightweight data structure called LightBlock to streamline broadcast content. Furthermore, we design a novel Unrelated Block Offloading Filter to avoid the unlimited growth of ledger without affecting blockchain's traceability. The extensive experiments demonstrate that LightChain can reduce the individual computational cost to 39.32% and speed up the block generation by up to 74.06%. In terms of storage and network usage, the reductions are 43.35% and 90.55%, respectively.

Electroencephalogram (EEG) that measures the electrical activity of the brain has been widely employed for diagnosing epilepsy which is one kind of brain abnormalities. With the advancement of low-cost wearable brain-computer interface devices, it is possible to monitor EEG for epileptic seizure detection in daily use. However, it is still challenging to develop seizure classification algorithms with a considerable higher accuracy and lower complexity. In this study, we propose a lightweight method which can reduce the number of features for a multiclass classification to identify three different seizure statuses (i.e., Healthy, Interictal and Epileptic seizure) through EEG signals with a wearable EEG sensors using Extended Correlation-Based Feature Selection (ECFS). More specifically, there are three steps in our proposed approach. Firstly, the EEG signals were segmented into five frequency bands and secondly, we extract the features while the unnecessary feature space was eliminated by developing the ECFS method. Finally, the features were fed into five different classification algorithms, including Random Forest, Support Vector Machine, Logistic Model Trees, RBF Network and Multilayer Perceptron. Experimental results have shown that Logistic Model Trees provides the highest accuracy of 97.6% comparing to other classifiers.

Motor learning is the ability to acquire a new motor skill, which plays an important role in rehabilitation as patients learn exercise programs or modify movements to regain pain free function. In this paper, we design an easy-to-use multimodal sensor system to assess motor learning. We developed a motor learning assessment device with a touch screen and Leap Motion to record the subject hand movement during a Serial Reaction Time Task(SRTT). The SRTT consists of upper limb reaching to targets in multi-dimensions. The device records metrics of time and movement efficiency and examines motor learning based on data analysis. This device can provide clinicians with data that can inform their approach to training. We recruited a total of 11 participants, with and without chronic pain to evaluate the device using a classifier model to assess participants' performance. The model shows our system works well to identify motor learning differences in individuals with and without chronic pain.

Stroke is a leading cause of long-term impairment, causing a fatality if not act upon in time. Home-based post-stroke rehabilitation plays an important role in helping patients to regain normal mobility and functionality at their residence. However, existing home-based rehabilitation approaches fail to effectively motivate patients on frequent engagement with exercise to achieve the intended outcome. In this paper, we develop VRInsole, a synthetical solution combining a Smart Insole footwear sensor and virtual reality (VR), targeting lower extremity mobility training in an immersive environment for stroke rehabilitation. Specifically, the motion information collected from the Smart Insole serve as the input for the VR to perform corresponding exercise animations. To prove the feasibility of VRInsole, an experiment is conducted on the recognition of lower extremity motion direction, which achieves an average accuracy of 93.9%.

Motor impairment is common following stroke. Diminished strength and coordination contribute to reduced ability to perform activities of daily living. The existing healthcare models focus on delivering rehabilitation during the first few months following stroke. Yet, to regain motor control to the greatest degree, rehabilitation should continue across the lifespan. Currently, individuals with stroke are responsible for self-managing their rehabilitation once therapist guided rehabilitation has concluded. Individuals with stroke are frequently given a written home exercise program to help guide their home rehabilitation, but poor compliance demonstrates a better approach is necessary. In this study, we propose BiGRA, a novel system to more effectively facilitate in-home bilateral rehabilitation. This system holds merits in: (1) An end-to-end task-oriented system for bilateral grip control which emphasizes the modulation of grip coordination between hands; and (2) Innovative metrics framework to quantitatively analyze the motor control performance. The evaluation shows that BiGRA can objectively measure the patients' task performance and is a promising assessment tool for stroke rehabilitation.

As 3D printing technology begins to outpace traditional manufacturing, malicious users increasingly have sought to leverage this widely accessible platform to produce unlawful tools for criminal activities. Therefore, it is of paramount importance to identify the origin of unlawful 3D printed products using digital forensics. Traditional countermeasures, including information embedding or watermarking, rely on supervised manufacturing process and are impractical for identifying the origin of 3D printed tools in criminal applications. We argue that 3D printers possess unique fingerprints, which arise from hardware imperfections during the manufacturing process, causing discrepancies in the line formation of printed physical objects. These variations appear repeatedly and result in unique textures that can serve as a viable fingerprint on associated 3D printed products. To address the challenge of traditional forensics in identifying unlawful 3D printed products, we present PrinTracker, the 3D printer identification system, which can precisely trace the physical object to its source 3D printer based on their fingerprint. Results indicate that PrinTracker provides a high accuracy using 14 different 3D printers. Under unfavorable conditions (e.g. restricted sample area, location and process), the PrinTracker can still achieve an acceptable accuracy of 92%. Furthermore, we examine the effectiveness, robustness, reliability and vulnerabilities of the PrinTracker in multiple real-world scenarios.

Nowadays, mobile devices have become an important part of our daily life. Numerous mobile sensing applications are enabled by various mobile platforms, which leverage machine learning techniques to detect or classify the events of interest such as human activities and health conditions. To achieve this, each user is required to provide a considerable amount of training samples. However, in practice, a large portion of the users may provide only a few or even zero labels, due to various reasons such as privacy concern or simply laziness. A straightforward solution to this problem is to gather the data of all the users in a central database, and train a global classifier from the combined data. Such global classifier, however, may not work well since it ignores the variety in different users' data. To address this challenge, we propose PLOS, a Personalized Learning framework for mObile Sensing applications. PLOS can jointly model the commonness shared among the users as well as the differences between them, which are inferred from both the label information and the underlying structures of individual data. We further develop the distributed PLOS where the raw data of the users are locally processed so that the users only need to send model parameters to the server. Through extensive experiments on both synthetic data and real mobile sensing systems, we show that the proposed PLOS framework is scalable and efficient in energy, computation, and communication costs, and can achieve more accurate classification results compared with the baseline methods.

Driven by a wide range of real-world applications, significant efforts have recently been made to explore device-free human activity recognition techniques that utilize the information collected by various wireless infrastructures to infer human activities without the need for the monitored subject to carry a dedicated device. Existing device free human activity recognition approaches and systems, though yielding reasonably good performance in certain cases, are faced with a major challenge. The wireless signals arriving at the receiving devices usually carry substantial information that is specific to the environment where the activities are recorded and the human subject who conducts the activities. Due to this reason, an activity recognition model that is trained on a specific subject in a specific environment typically does not work well when being applied to predict another subject's activities that are recorded in a different environment. To address this challenge, in this paper, we propose EI, a deep-learning based device free activity recognition framework that can remove the environment and subject specific information contained in the activity data and extract environment/subject-independent features shared by the data collected on different subjects under different environments. We conduct extensive experiments on four different device free activity recognition testbeds: WiFi, ultrasound, 60 GHz mmWave, and visible light. The experimental results demonstrate the superior effectiveness and generalizability of the proposed EI framework.

In recent years, biometric techniques (e.g., fingerprint or iris) are increasingly integrated into mobile devices to offer security advantages over traditional practices (e.g., passwords and PINs) due to their ease of use in user authentication. However, existing biometric systems are with controversy: once divulged, they are compromised forever - no one can grow a new fingerprint or iris. This work explores a truly cancelable brain-based biometric system for mobile platforms (e.g., smart headwear). Specifically, we present a new psychophysiological protocol via non-volitional brain response for trustworthy mobile authentication, with an application example of smart headwear. Particularly, we address the following research challenges in mobile biometrics with a theoretical and empirical combined manner: (1) how to generate reliable brain responses with sophisticated visual stimuli; (2) how to acquire the distinct brain response and analyze unique features in the mobile platform; (3) how to reset and change brain biometrics when the current biometric credential is divulged. To evaluate the proposed solution, we conducted a pilot study and achieved an f -score accuracy of 95.46% and equal error rate (EER) of 2.503%, thereby demonstrating the potential feasibility of neurofeedback based biometrics for smart headwear. Furthermore, we perform the cancelability study and the longitudinal study, respectively, to show the effectiveness and usability of our new proposed mobile biometric system. To the best of our knowledge, it is the first in-depth research study on truly cancelable brain biometrics for secure mobile authentication.

No abstract available.

While malicious attacks on electronic devices (e-devices) have become commonplace, the use of e-devices themselves for malicious attacks has increased (e.g., explosives and eavesdropping). Modern e-devices (e.g., spy cameras, bugs or concealed weapons) can be sealed in parcels/boxes, hidden under clothing or disguised with cardboard to conceal their identities (named as hidden e-devices hereafter), which brings challenges in security screening. Inspection equipment (e.g., X-ray machines) is bulky and expensive. Moreover, screening reliability still rests on human performance, and the throughput in security screening of passengers and luggages is very limited. To this end, we propose to develop a low-cost and practical hidden e-device recognition technique to enable efficient screenings for threats of hidden electronic devices in daily life. First, we investigate and model the characteristics of nonlinear effects, a special passive response of electronic devices under millimeter-wave (mmWave) sensing. Based on this theory and our preliminary experiments, we design and implement, E-Eye, an end-to-end portable hidden electronics recognition system. E-Eye comprises a low-cost (i.e., under $100), portable (i.e., 11.8cm by 4.5cm by 1.8cm) and light-weight (i.e., 45.5g) 24GHz mmWave probe and a smartphone-based e-device recognizer. To validate the E-Eye performance, we conduct experiments with 46 commodity electronic devices under 39 distinct categories. Results show that E-Eye can recognize hidden electronic devices in parcels/boxes with an accuracy of more than 99% and has an equal error rate (EER) approaching 0.44% under a controlled lab setup. Moreover, we evaluate the reliability, robustness and performance variation of E-Eye under various real-world circumstances, and E-Eye can still achieve accuracy over 97%. Intensive evaluation indicates that E-Eye is a promising solution for hidden electronics recognition in daily life.

The rapid development of Internet of Things is yielding a huge volume of time series data, the real-time mining of which becomes a major load for data centers. The computation bottleneck in time series mining is the distance measure, in which dynamic time warping (DTW) is one of the most widely used distance measures. Recently, various software optimization and hardware acceleration techniques have been proposed for DTW acceleration. However, the throughput and energy efficiency of DTW are still big concerns considering the ever-increasing volume of times series. In this paper, we propose a high-throughput and efficient memristor-based DTW architecture for real-time time series mining on data centers. Specifically, memristors have been adopted for both computation and configuration of the computing architecture. The computation flow in this architecture is fully presented in a continuous and asynchronous manner. To improve the computation efficiency, we propose an early lower bound algorithm by exploiting the predictability in the circuit characteristic. Experiments are performed with module evaluation and end-to-end evaluation including three popular applications: 1) similarity search; 2) classification; and 3) anomaly detection. Experimental results indicate that, compared to existing approaches, the speedup and energy efficiency improvement are 12x-43x and 51x-287x, respectively.

As a micro-engineered biomimetic system to replicate key functions of living organs, organ-on-a-chip (OC) technology provides a high-throughput model for investigating complex cell interactions with both high temporal and spatial resolutions in biological studies. Typically, microscopy and high-speed video cameras are used for data acquisition, which are expensive and bulky. Recently, compressed sensing (CS) has increasingly attracted attentions due to its extremely low-complexity structure and low sampling rate. However, there is no CS solution tailored for tempo-spatial information acquisition. In this paper, we propose tempo-spatial CS (TS-CS), a unified CS architecture for OC stream, which achieves significant cost reduction and truly combines sensing with compression along the temporal and spatial domains. We point out that TS-CS can consistently achieve better performance by exploiting tempo-spatial compressibility in OC data. To this end, we comprehensively evaluate the system performance by employing four different bases for CS. With comparison to the traditional way, we show that TS-CS always obtains better recovery result with a throughput bound and can achieve around throughput improvement under a reconstruction demand by applying discrete cosine transform matrix as the basis.

Recently, continuous 3D printing, a revolutionary branch of legacy additive manufacturing, has made its two-order time efficiency breakthrough in industrial manufacturing. As its manufacturing technique advances rapidly, the prefabrication to slice the 3D object into image layers becomes potential to impede further improvement of production efficiency. In this paper, we present two scalable and efficient graphic processing unit (GPU) enabled schemes, i.e., pixelwise parallel slicing and fully parallel slicing, to accelerate the image-projection based slicing algorithm in continuous 3D printing. Specifically, the pixelwise approach utilizes the pixel-level parallelism and exploits the in-shared-memory computing on GPU. The fully parallel method aggressively expands the parallelism on both triangle mesh size and slicing layers. The thread-level priority competing issue, resulting from full parallelism, is addressed by a critical area using atomic operation. Experiments with real 3D object benchmarks show that our pixelwise parallel slicing can gain one order of magnitude runtime reduction to CPU, and the fully parallel slicing achieves two orders improvement. We also evaluate the scalability of both proposed schemes.

As the next-generation manufacturing driven force, 3D printing technology is having a transformative effect on various industrial domains and has been widely applied in a broad spectrum of applications. It also progresses towards other versatile fields with portable battery-powered 3D printers working on a limited energy budget. While reducing manufacturing energy is an essential challenge in industrial sustainability and national economics, this growing trend motivates us to explore the energy consumption of the 3D printer for the purpose of energy efficiency. To this end, we perform an in-depth analysis of energy consumption in commercial, off-the-shelf 3D printers from an instruction-level perspective. We build an instruction-level energy model and an energy profiler to analyze the energy cost during the fabrication process. From the insights obtained by the energy profiler, we propose and implement a cross-layer energy optimization solution, called 3DGates, which spans the instruction-set, the compiler and the firmware. We evaluate 3DGates over 338 benchmarks on a 3D printer and achieve an overall energy reduction of 25%.

The rapid development of Internet-of-Things (IoT) is yielding a huge volume of time series data, the real-time mining of which becomes a major load for data centers. The computation bottleneck in time series data mining is the distance function, which has been tackled by various software optimization and hardware acceleration techniques recently. However, each of these techniques is only designed or optimized for a specific distance function. To address this problem, in this paper we propose an efficient and reconfigurable memristor-based distance accelerator for real-time and energy-efficient data mining with time series on data centers. Common circuit structure is extracted to save chip areas, and the circuit can be configured to any specific distance functions. Experimental results show that compared with existing works, our work has achieved a speedup of 3.5x-376x on performance and an improvement of 1-3 orders of magnitude on energy efficiency.

Wearable computing systems have spurred many opportunities to continuously monitor human bodies with sensors worn on or implanted in the body. These emerging platforms have started to revolutionize many fields, including healthcare and wellness applications, particularly when integrated with intelligent analytic capabilities. However, a significant challenge that computer architects are facing is how to embed sophisticated analytic capabilities in wearable computers in an energy-efficient way while not compromising system performance. In this paper, we present XPro, a novel cross-end analytic engine architecture for wearable computing systems. The proposed cross-end architecture is able to realize a generic classification design across wearable sensors and a data aggregator with high energy-efficiency. To facilitate the practical use of XPro, we also develop an Automatic XPro Generator that formally generates XPro instances according to specific design constraints. As a proof of concept, we study the design and implementation of XPro with six different health applications. Evaluation results show that, compared with state-of-the-art methods, XPro can increase the battery life of the sensor node by 1.6-2.4X while at the same time reducing system delay by 15.6-60.8% for wearable computing systems.

Continuous authentication is of great importance to maintain the security level of a system throughout the login session. The goal of this work is to investigate a trustworthy, continuous, and non-contact user authentication approach based on a heart-related biometric that works in a daily-life environment. To this end, we present a novel, continuous authentication system, namely Cardiac Scan, based on geometric and non-volitional features of the cardiac motion. Cardiac motion is an automatic heart deformation caused by self-excitement of the cardiac muscle, which is unique to each user and is difficult (if not impossible) to counterfeit. Cardiac Scan features intrinsic liveness detection, unobtrusiveness, cost-effectiveness, and high usability. We prototype a remote, high-resolution cardiac motion sensing system based on the smart DC-coupled continuous-wave radar. Fiducial-based invariant identity descriptors of cardiac motion are extracted after the radar signal demodulation. We conduct a pilot study with 78 subjects to evaluate Cardiac Scan in accuracy, authentication time, permanence, evaluation in complex conditions, and vulnerability. Specifically, Cardiac Scan achieves 98.61% balanced accuracy (BAC) and 4.42% equal error rate (EER) in a real-world setup. We demonstrate that Cardiac Scan is a robust and usable continuous authentication system.

Inkjet 3D printing is a disruptive manufacturing technology in emerging metal- and bio-printing applications. The nozzle of the printer deposits tiny liquid droplets, which are subsequently solidified on a target location. Due to the elegant concept of micro-droplet deposition, inkjet 3D printing is capable of achieving a sub-millimeter scale manufacturing resolution. However, the droplet deposition process is dynamic and uncertain which imposes a significant challenge on quality assurance of inkjet 3D printing in terms of product reproducibility and process repeatability. To this end, we present Luban as a certification tool to examine the printing quality in the inkjet printing process. Luban is a new low-cost and in-situ droplet micro-sensing system that can precisely detect, analyze and localize a droplet. Specifically, we present a novel tiny object sensing method by exploiting the computational light beam field and its sensitive interference effect. The realization of Luban is associated with two technical thrusts. First, we study integral sensing, i.e., a new scheme towards computational light beam field sensing, to efficiently extract droplet location information. This sensing scheme offers a new in-situ droplet sensing modality, which can promote the information acquisition efficiency and reduce the sensing cost compared to prior approaches. Second, we characterize interference effect of the computational light beam field and develop an efficient integration-domain droplet location estimation algorithm. We design and implement Luban in a real inkjet 3D printing system with commercially off-the-shelf devices, which costs less than a hundred dollars. Experimental results in both simulation and real-world evaluation show that Luban can reach the certification precision of a sub-millimeter scale with a 99% detection accuracy of defect droplets; furthermore, the enabled in-situ certification throughput is as high as over 700 droplets per second. Therefore, the performance of our Luban system can meet the quality assurance requirements (e.g., cost-effective, in-situ, high-accuracy and high-throughput) in general industrial applications.

Nurses regularly perform patient handling activities. These activities with awkward postures expose healthcare providers to a high risk of overexertion injury. The recognition of patient handling activities is the first step to reduce injury risk for caregivers. The current practice on workplace activity recognition is based on human observational approach, which is neither accurate nor projectable to a large population. In this paper, we aim at addressing these challenges. Our solution comprises a smart wearable device and a novel spatio-temporal warping (STW) pattern recognition framework. The wearable device, named Smart Insole 2.0, is equipped with a rich set of sensors and can provide an unobtrusive way to automatically capture the information of patient handling activities. The STW pattern recognition framework fully exploits the spatial and temporal characteristics of plantar pressure by calculating a novel warped spatio-temporal distance, to quantify the similarity for the purpose of activity recognition. To validate the effectiveness of our framework, we perform a pilot study with eight subjects, including eight common activities in a nursing room. The experimental results show the overall classification accuracy achieves 91.7%. Meanwhile, the qualitative profile and load level can also be classified with accuracies of 98.3% and 92.5%, respectively.

Additive manufacturing, also known as 3D printing, has been increasingly applied to fabricate highly intellectual property (IP) sensitive products. However, the related IP protection issues in 3D printers are still largely underexplored. On the other hand, smartphones are equipped with rich onboard sensors and have been applied to pervasive mobile surveillance in many applications. These facts raise one critical question: is it possible that smartphones access the side-channel signals of 3D printer and then hack the IP information? To answer this, we perform an end-to-end study on exploring smartphone-based side-channel attacks against 3D printers. Specifically, we formulate the problem of the IP side-channel attack in 3D printing. Then, we investigate the possible acoustic and magnetic side-channel attacks using the smartphone built-in sensors. Moreover, we explore a magnetic-enhanced side-channel attack model to accurately deduce the vital directional operations of 3D printer. Experimental results show that by exploiting the side-channel signals collected by smartphones, we can successfully reconstruct the physical prints and their G-code with Mean Tendency Error of 5.87% on regular designs and 9.67% on complex designs, respectively. Our study demonstrates this new and practical smartphone-based side channel attack on compromising IP information during 3D printing.

As mobile technology grows rapidly, the smartphone has become indispensable for transmitting private user data, storing the sensitive corporate files, and conducting secure payment transactions. However, with mobile security research lagging, smartphones are extremely vulnerable to unauthenticated access. In this paper, we present, EyeVeri, a novel eye-movement-based authentication system for smartphone security protection. Specifically, EyeVeri tracks human eye movement through the built-in front camera and applies the signal processing and pattern matching techniques to explore volitional and non-volitional gaze patterns for access authentication. Through a comprehensive user study, EyeVeri performs well and is a promising approach for smartphone user authentication. We also discuss the evaluation results in-depth and analyze opportunities for future work.

In recent years, the analog-to-information converter (AIC), based on compressed sensing (CS) paradigm, is a promising solution to overcome the performance and energy-efficiency limitations of traditional analog-to-digital converters (ADC). Especially, AIC can enable sub-Nyquist signal sampling proportional to the intrinsic information in biomedical applications. However, the legacy AIC structure is tailored toward specific applications, which lacks of flexibility and prevents its universality. In this paper, we introduce a novel programmable AIC architecture, Pro-AIC, to enable effective configurability and reduce its energy overhead by integrating efficient multiplexing hardware design. To improve the quality and time-efficiency of Pro-AIC configuration, we also develop a rapid configuration algorithm, called RapSpiral, to quickly find the near-optimal parameter configuration in Pro-AIC architecture. Specifically, we present a design metric, trade-off penalty, to quantitatively evaluate the performance-energy trade-off. The RapSpiral controls a penalty-driven shrinking triangle to progressively approximate to the optimal trade-off. Our proposed RapSpiral is with log(n) complexity yet high accuracy, without pretraining and complex parameter tuning procedure. RapSpiral is also probable to avoid the local minimum pitfalls. Experimental results indicate that our RapSpiral algorithm can achieve more than 30x speedup compared with the brute force algorithm, with only about 3% trade-off compromise to the optimum in Pro-AIC. Furthermore, the scalability is also verified on larger size benchmarks.

Gait analysis is an important medical diagnostic process and has many applications in healthcare, rehabilitation, therapy, and exercise training. However, typical gait analysis has to be performed in a gait laboratory, which is inaccessible for a large population and cannot provide natural gait measures. In this paper, we present a novel sensor device, namely, Smart Insole, to tackle the challenge of efficient gait monitoring in real life. An array of electronic textile (eTextile)-based pressure sensors are integrated in the insole to fully measure the plantar pressure. Smart Insole is also equipped with a low-cost inertial measurement unit including a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer to capture the gait characteristics in motion. Smart Insole can offer precise acquisition of gait information. Meanwhile, it is lightweight, thin, and comfortable to wear, providing an unobtrusive way to perform the gait monitoring. Furthermore, a smartphone graphic user interface is developed to display the sensor data in real-time via Bluetooth low energy. We perform a set of experiments in four real-life scenes including hallway walking, ascending/descending stairs, and slope walking, where gait parameters and features are extracted. Finally, the limitation and improvement, wearability and usability, further work, and healthcare-related potential applications are discussed.

The past decades have witnessed a rapid surge in new sensing and monitoring devices for well-being and healthcare. One key representative in this field is body sensor networks (BSNs). However, with advances in sensing technologies and embedded systems, wireless communication has gradually become one of the dominant energy-consuming sectors in BSN applications. Recently, compressed sensing (CS) has attracted increasing attention in solving this problem due to its enabled sub-Nyquest sampling rate. In this paper, we investigate the quantization effect in CS architecture and argue that the quantization configuration is a critical factor of the energy efficiency for the entire CS architecture. To this end, we present a novel configurable quantized compressed sensing (QCS) architecture, in which the sampling rate and quantization are jointly explored for better energy efficiency. Furthermore, to combat the computational complexity of the configuration procedure, we propose a rapid configuration algorithm, called RapQCS. According to the experiments involving several categories of real biosignals, the proposed configurable QCS architecture can gain more than 66% performance-energy tradeoff than the fixed QCS architecture. Moreover, our proposed RapQCS algorithm can achieve over 150× speedup on average, while decreasing the reconstructed signal fidelity by only 2.32%.

The increasing popularity of mobile devices has brought severe challenges to device usability and big data analysis. In this paper we investigate the intellectual recommender system on cell phones by incorporating mobile data analysis. Nowadays with the development of smart phones, more and more applications have emerged on various areas, such as entertainment, education and health care. While these applications have brought great convenience to people's daily life, they also provide tremendous opportunities for analyzing users' interests. In this work we develop an Android background service to collect the user behaviors and analyze their preferences based on their Android application usage. As one of the most intuitive media for visual representation, videos with various types of contents are recommended to users based on a proposed graphical model. The proposed model jointly utilizes the textual descriptions of Android applications and videos, as well as the extracted video content based features. Besides, by analyzing the user's habit of application usage we seamlessly integrate the user's personal interests during the recommendation. The extensive comparisons to multiple baselines reveal the superiority of the proposed model on the recommendation quality. Furthermore, we conduct experiments on personalized recommendation to demonstrate the capacity of the proposed model in effectively analyzing the user's personal interests.

Sleep monitoring is receiving increased attention in the healthcare community, because the quality of sleep has a great impact on human health. Existing in-home sleep monitoring devices are either obtrusive to the user or cannot provide adequate sleep information. To this end, we present SleepSense, a contactless and low-cost sleep monitoring system for home use that can continuously detect the sleep event. Specifically, SleepSense consists of three parts: an electromagnetic probe, a robust automated radar demodulation module, and a signal processing framework for sleep event recognition, including on-bed movement, bed exit, and breathing event. We present a prototype of the SleepSense system, and perform a set of comprehensive experiments to evaluate the performance of sleep monitoring. Using a real-case evaluation, experimental results indicate that SleepSense can perform effective sleep event detection and recognition in practice.

Wireless sensor nodes advance the brain-computer interface (BCI) from laboratory setup to practical applications. Compressed sensing (CS) theory provides a sub-Nyquist sampling paradigm to improve the energy efficiency of electroencephalography (EEG) signal acquisition. However, EEG is a structure-variational signal with time-varying sparsity, which decreases the efficiency of compressed sensing. In this paper, we present a new adaptive CS architecture to tackle the challenge of EEG signal acquisition. Specifically, we design a dynamic knob framework to respond to EEG signal dynamics, and then formulate its design optimization into a dynamic programming problem. We verify our proposed adaptive CS architecture on a publicly available data set. Experimental results show that our adaptive CS can improve signal reconstruction quality by more than 70% under different energy budgets while only consuming 187.88 nJ/event. This indicates that the adaptive CS architecture can effectively adapt to the EEG signal dynamics in the BCI.

Remote health monitoring (RHM) has emerged as a solution to help reduce the cost burden of unhealthy lifestyles and aging populations. Enhancing compliance to prescribed medical regimens is an essential challenge to many systems, even those using smartphone technology. In this paper, we provide a technique to improve smartphone battery consumption and examine the effects of smartphone battery lifetime on compliance, in an attempt to enhance users' adherence to remote monitoring systems. We deploy WANDA-CVD, an RHM system for patients at risk of cardiovascular disease (CVD), using a wearable smartphone for detection of physical activity. We tested the battery optimization technique in an in-lab pilot study and validated its effects on compliance in the Women's Heart Health Study. The battery optimization technique enhanced the battery lifetime by 192% on average, resulting in a 53% increase in compliance in the study. A system like WANDA-CVD can help increase smartphone battery lifetime for RHM systems monitoring physical activity.

The ability to continuously monitor respiration rates of patients in homecare or in clinics is an important goal. Past research showed that monitoring patient breathing can lower the associated mortality rates for long-term bedridden patients. Nowadays, in-bed sensors consisting of pressure sensitive arrays are unobtrusive and are suitable for deployment in a wide range of settings. Such systems aim to extract respiratory signals from time-series pressure sequences. However, variance of movements, such as unpredictable extremities activities, affect the quality of the extracted respiratory signals. BreathSens, a high-density pressure sensing system made of e-Textile, profiles the underbody pressure distribution and localizes torso area based on the high-resolution pressure images. With a robust bodyparts localization algorithm, respiratory signals extracted from the localized torso area are insensitive to arbitrary extremities movements. In a study of 12 subjects, BreathSens demonstrated its respiratory monitoring capability with variations of sleep postures, locations, and commonly tilted clinical bed conditions.

We present a low-cost medical embedded system for detecting diabetic peripheral neuropathy (DPN). More specifically, we use low-cost infrared sensors to capture in a non-contact fashion the thermal response of the feet following cold stimulation. The rate of thermal regulation is then calculated based on temperature recovery and used to quantify the degree of the foot sensation. This device and methodology enables early detection of risk for DPN for in-home use or at family care clinics where fancy, expensive tests cannot be conducted.

Statistical circuit simulation is exhibiting increasing importance for circuit design under process variations. Existing approaches cannot efficiently analyze the failure probability for circuits with a large number of variation, nor handle problems with multiple disjoint failure regions. The proposed rare event microscope (REscope) first reduces the problem dimension by pruning the parameters with little contribution to circuit failure. Furthermore, we applied a nonlinear classifier which is capable of identifying multiple disjoint failure regions. In REscope, only likely-to-fail samples are simulated then matched to a generalized pareto distribution. On a 108-dimension charge pump circuit in PLL design, REscope outperforms the importance sampling and achieves more than 2 orders of magnitude speedup compared to Monte Carlo. Moreover, it accurately estimates failure rate, while the importance sampling totally fails because failure regions are not correctly captured.

The significance of FPGA test and the challenge of its increasing cost can never be ignored. In island-style FPGA architectures, hex lines are the principal interconnect resources. Testing hex lines and hex Programmable Interconnect Points (PIPs) have remained as the major technical difficulty in FPGAs test due to complex interconnect rules. Particularly, test in oblique direction of hex PIPs has rarely been addressed in previous studies. Towards this challenge, this paper for the first time proposes a coordinate system and formulates the interconnect rules of hex lines as mathematical equations. For hex PIPs in horizontal and vertical direction, an efficient circle test structure is formed by coordinate equations. For hex PIPs in oblique direction, the coordinate method is used to generate the partial-cascade pattern. The corresponding test vector is also generated, which ensures the ergodicity of hex PIPs in oblique direction. In addition to hex PIPs, hex lines are also covered without extra effort. Compared to previous researches, the configuration number for hex lines is decreased significantly. We evaluate this method on Xilinx XC2V1000, and experimental results show that our proposed method achieves 100% fault coverage for hex PIPs and can be generally applied to all mainstream island-style FPGAs with a similar interconnect structure currently.

Mobile Health (mHealth) is expected to play a special role in today and the future healthcare delivery. Based on this trend, we design a Smart Wristlet mHealth system with mobile interface. The designed Smart Wristlet is dedicated to offer real-time alert for elderly fall, which is the most important when population ageing is becoming. In the Smart Wristlet mHealth system, fall detection is the “bottleneck” of the system operation. To remove this bottleneck away, we propose a fall perception solution for elderly care. In this proposal, we abstract and construct primitive-based features from raw data collected by the Smart Wristlet mHealth system, in which the most valuable features can be selected by using a TF-IDF (Term Frequency-Inverse Document Frequency) metric. In reality, these selected features are the most effective to perform fall detection. Our system tests and clinical trials demonstrate that this proposal is eligible to turn the Smart Wristlet mHealth system into a real solution for elderly care. Results show that the recognition precision and recall can reach 93% and 88%, respectively. Compared with existing solutions, the gain from our proposal is an efficient prevention method for elderly fall, and can save more than 800 million dollars per year at today's socio-economic level.

Detecting human activity independent of intensity is essential in many applications, primarily in calculating metabolic equivalent rates and extracting human context awareness. Many classifiers that train on an activity at a subset of intensity levels fail to recognize the same activity at other intensity levels. This demonstrates weakness in the underlying classification method. Training a classifier for an activity at every intensity level is also not practical. In this paper, we tackle a novel intensity-independent activity recognition problem where the class labels exhibit large variability, the data are of high dimensionality, and clustering algorithms are necessary. We propose a new robust stochastic approximation framework for enhanced classification of such data. Experiments are reported using two clustering techniques, K-Means and Gaussian Mixture Models. The stochastic approximation algorithm consistently outperforms other well-known classification schemes which validate the use of our proposed clustered data representation. We verify the motivation of our framework in two applications that benefit from intensity-independent activity recognition. The first application shows how our framework can be used to enhance energy expenditure calculations. The second application is a novel exergaming environment aimed at using games to reward physical activity performed throughout the day, to encourage a healthy lifestyle.

Physical rehabilitation is an important process for patients recovering after surgery. In this paper, we propose and develop a framework to monitor on-bed range of motion exercises that allows physical therapists to evaluate patient adherence to set exercise programs. Using a dense pressure sensitive bedsheet, a sequence of pressure maps are produced and analyzed using manifold learning techniques. We compare two methods, Local Linear Embedding and Isomap, to reduce the dimensionality of the pressure map data. Once the image sequences are converted into a low dimensional manifold, the manifolds can be compared to expected prior data for the rehabilitation exercises. Furthermore, a measure to compare the similarity of manifolds is presented along with experimental results for five on-bed rehabilitation exercises. The evaluation of this framework shows that exercise compliance can be tracked accurately according to prescribed treatment programs.

Cardiovascular disease (CVD) is a major issue to public health. It contributes 41% to the Chinese death rate each year. This huge loss encouraged us to develop a Wearable Efficient teleCARdiology systEm (WE-CARE) for early warning and prevention of CVD risks in real time. WE-CARE is expected to work 24/7 online for mobile health (mHealth) applications. Unfortunately, this purpose is often disrupted in system experiments and clinical trials, even if related enabling technologies work properly. This phenomenon is rooted in the overload issue of complex Electrocardiogram (ECG) data in terms of system integration. In this study, our main objective is to get a system light-loading technology to enable mHealth with a benchmarked ECG anomaly recognition rate. To achieve this objective, we propose an approach to purify clinical features from ECG raw data based on manifold learning, called the Manifold-based ECG-feature Purification algorithm. Our clinical trials verify that our proposal can detect anomalies with a recognition rate of up to 94% which is highly valuable in daily public health-risk alert applications based on clinical criteria. Most importantly, the experiment results demonstrate that the WE-CARE system enabled by our proposal can enhance system reliability by at least two times and reduce false negative rates to 0.76%, and extend the battery life by 40.54%, in the system integration level.

Detecting human activity independent of intensity is essential in many applications, primarily in calculating metabolic equivalent rates (MET) and extracting human context awareness from on-body inertial sensors. Many classifiers that train on an activity at a subset of intensity levels fail to classify the same activity at other intensity levels. This demonstrates weakness in the underlying activity model. Training a classifier for an activity at every intensity level is also not practical. In this paper we tackle a novel intensity-independent activity recognition application where the class labels exhibit large variability, the data is of high dimensionality, and clustering algorithms are necessary. We propose a new robust Stochastic Approximation framework for enhanced classification of such data. Experiments are reported for each dataset using two clustering techniques, K-Means and Gaussian Mixture Models. The Stochastic Approximation algorithm consistently outperforms other well-known classification schemes which validates the use of our proposed clustered data representation.

This paper presents the design of an on-bed rehabilitation exercise monitoring system that utilizes a high density sensor bedsheet to evaluate active range of motion exercises. We propose and develop a novel framework to analyze the progression of pressure image sequences using manifold learning. The image sequences are reduced to a low dimensional subspace that can be measured against expected prior data for each of the rehabilitation exercises. We also present a metric to compare manifold similarities. Our experimental results on five on-bed exercises show that this system can accurately track compliance of patients to prescribed treatment programs. It allows physical therapists to evaluate how well patients adhere to the rehabilitation exercises. The system is convenient to setup, unobtrusive, and can be used for reliable, long term monitoring.

This paper presents the design of an on-bed rehabilitation exercise monitoring system that utilizes a high density sensor bedsheet to evaluate active range of motion exercises. We propose and develop a novel framework to analyze the progression of pressure image sequences using manifold learning. The image sequences are reduced to a low dimensional subspace that can be measured against expected prior data for each of the rehabilitation exercises. We also present a metric to compare manifold similarities. Our experimental results on five on-bed exercises show that this system can accurately track compliance of patients to prescribed treatment programs. It allows physical therapists to evaluate how well patients adhere to the rehabilitation exercises. The system is convenient to setup, unobtrusive, and can be used for reliable, long term monitoring.

Compressive sensing (CS) is a promising technology for the low-power and cost-effective data acquisition in wireless healthcare systems. However, its efficient realtime signal reconstruction is still challenging, and there is a clear demand for hardware acceleration. In this paper, we present the first single-precision floating-point CS reconstruction engine implemented a Kintex-7 FPGA using the orthogonal matching pursuit (OMP) algorithm. In order to achieve high performance with maximum hardware utilization, we propose a highly parallel architecture that shares the computing resources among different tasks of OMP by using configurable processing elements (PEs). By fully utilizing the FPGA recourses, our implementation has 128 PEs in parallel and operates at 53.7 MHz. In addition, it can support 2x larger problem size and 10x more sparse coefficients than prior work, which enables higher reconstruction accuracy by adding finer details to the recovered signal. Hardware results from the ECG reconstruction tests show the same level of accuracy as the double-precision C program. Compared to the execution time of a 2.27 GHz CPU, the FPGA reconstruction achieves an average speed-up of 41x.

Sleep plays a pivotal role in the quality of life, and sleep posture is related to many medical conditions such as sleep apnea. In this paper, we design a dense pressure-sensitive bedsheet for sleep posture monitoring. In contrast to existing techniques, our bedsheet system offers a completely unobtrusive method using comfortable textile sensors. Based on high-resolution pressure distributions from the bedsheet, we develop a novel framework for pressure image analysis to monitor sleep postures, including a set of geometrical features for sleep posture characterization and three sparse classifiers for posture recognition. We run a pilot study and evaluate the performance of our methods with 14 subjects to analyze 6 common postures. The experimental results show that our proposed method enables reliable sleep posture recognition and offers better overall performance than state-of-the-art methods, achieving up to 83.0% precision and 83.2% recall on average.

ECG analysis is universal and important in miscellaneous medical applications. However, high computation complexity is a problem which has been shown in several levels of conventional data mining algorithms for ECG analysis. In this paper, we presented a novel manifold approach to visualize and analyze the ECG signal. According to regularity of the data, our algorithm can discover the intrinsic structure and represent the streaming data with a 1-D manifold on a 2-D space. Furthermore, the proposed algorithm can reliably detect the anomaly in ECG streaming data. We evaluated the performance of the algorithm with two different anomalies in wearable applications: for the anomaly from heart disorders such as apnea, arrythmia, our algorithm could achieve up to 90% recognition rate, for the anomaly from the ECG device, our algorithm could detect the outlier with 100%.

Human activity recognition using wearable body sensors is playing a significant role in ubiquitous and mobile computing. One of the issues related to this wearable technology is that the captured activity signals are highly dependent on the location where the sensors are worn on the human body. Existing research work either extracts location information from certain activity signals or takes advantage of the sensor location information as a priori to achieve better activity recognition performance. In this paper, we present a compressed sensing-based approach to co-recognize human activity and sensor location in a single framework. To validate the effectiveness of our approach, we did a pilot study for the task of recognizing 14 human activities and 7 on body-locations. On average, our approach achieves an 87:72% classification accuracy (the mean of precision and recall).

3-D vision technologies are extensively used in a wide variety of applications. Particularly glasses-based 3-D technology facilities are increasingly becoming affordable to our daily lives. Considering health issues raised by 3-D video technologies, to the best of our knowledge, most of the pilot studies are practiced in a highly-controlled laboratory environment only. In this paper, we present NeuroGlasses, a nonintrusive wearable physiological signal monitoring system to facilitate health analysis and diagnosis of 3-D video watchers. The NeuroGlasses system acquires health-related signals by physiological sensors and provides feedbacks of health-related features. Moreover, the NeuroGlasses system employs signal-specific reconstruction and feature extraction to compensate the distortion of signals caused by variation of the placement of the sensors. We also propose a server-based NeuroGlasses infrastructure where physiological features can be extracted for real-time response or collected on the server side for long term analysis and diagnosis. Through an on-campus pilot study, the experimental results show that NeuroGlasses system can effectively provide physiological information for healthcare purpose. Furthermore, it approves that 3-D vision technology has a significant impact on the physiological signals, such as EEG, which potentially leads to neural diseases.

Process variation in nanometer technology is becoming an important issue for cutting-edge FPGAs with a multimillion gate capacity. Considering both die-to-die and within-die variations in effective channel length, threshold voltage, and gate oxide thickness, we first develop closed-form models of chip-level FPGA leakage and timing variations. Experiments show that the mean and standard deviation computed by our models are within 3% from those computed by Monte Carlo simulation. We also observe that the leakage and timing variations can be up to 3X and 1.9X, respectively. We then derive analytical yield models considering both leakage and timing variations, and use such models to evaluate the performance of FPGA device and architecture considering process variations. Compared to the baseline, which uses the VPR architecture and device setup based on the ITRS roadmap, device and architecture tuning improves leakage yield by 10.4%, timing yield by 5.7%, and leakage and timing combined yield by 9.4%. We also observe that LUT size of 4 gives the highest leakage yield, LUT size of 7 gives the highest timing yield, but LUT size of 5 achieves the maximum leakage and timing combined yield. To the best of our knowledge, this is the first in-depth study on FPGA architecture and device coevaluation considering process variation.

The most challenging problem in the current block-based statistical static timing analysis (SSTA) is how to handle the max operation efficiently and accurately. Existing SSTA techniques suffer from limited modeling capability by using a linear delay model with Gaussian distribution, or have scalability problems due to expensive operations involved to handle non-Gaussian variation sources or nonlinear delays. To overcome these limitations, we propose efficient algorithms to handle the max operation in SSTA with both quadratic delay dependency and non-Gaussian variation sources simultaneously. Based on such algorithms, we develop an SSTA flow with quadratic delay model and non-Gaussian variation sources. All the atomic operations, max and add, are calculated efficiently via either closed-form formulas or low dimension (at most 2-D) lookup tables. We prove that the complexity of our algorithm is linear in both variation sources and circuit sizes, hence our algorithm scales well for large designs. Compared to Monte Carlo simulation for non-Gaussian variation sources and nonlinear delay models, our approach predicts the mean, standard deviation and 95% percentile point with less than 2% error, and the skewness with less than 10% error.

Sitting posture analysis is critical for daily applications in biomedical, education and healthcare fields. However, it remains unclear how to monitor sitting posture economically and comfortably. To this end, we presented an eTextile device, called eCushion, in this paper, which can analyze the sitting posture of human being accurately and non-invasively. First, we discussed the implementation of eCushion and design challenges of sensing data, such as scale, offset, rotation and crosstalk. Then, several effective techniques have been proposed to improve the recognition rate of sitting posture. Our experimental results show that the recognition rate of our eCushion system could achieve 92% for object-oriented cases and 79% for general cases.

We target body-wearable sensor networks, in which sensor nodes are strategically placed on the human body and the wireless communications are conducted on/off the surface of the body. The results, obtained by performing multiple experiments in outdoor environments, are presented. A single on body transmitter communicates with a single receiver node, which is located on the body or off the body at various distances. Sensor nodes utilized in our experiments are equipped with XBee and XBee Pro wireless modules for on body and off body communications, respectively. The focus of our work is to observe how the Received Signal Strength (RSS) and the Packet Reception Rate (PRR) vary as we change the communication distance and transmission power level. Our experimental results can be used to perform transmission power control with high precision in order not to exceed a certain packet error rate.

Field Programmable Gate Arrays (FPGAs) are very popular in today's embedded systems design, and Partial Runtime-Reconfigurable (PRTR) FPGAs allow HW tasks to be placed and removed dynamically at runtime. Hardware task scheduling on PRTR FPGAs brings many challenging issues to traditional real-time scheduling theory, which have not been adequately addressed by the research community compared to software task scheduling on CPUs. In this article, we consider the schedulability analysis problem of HW task scheduling on PRPR FPGAs. We derive utilization bounds for several variants of global preemptive/nonpreemptive EDF scheduling, and compare the performance of different utilization bound tests.