EDBT 2026 Demo / reviewers in the wild / expert
Yang Liu 0101
dblp:51/3710-101
· DBLP profile ↗
27ranked-venue papers
6as first author
18since 2021 · last 2026
0000-0002-2474-2004ORCID · conflict
Domains — the database's venue-derived domains; a paper can count in several
Computer networks · 18 · 5 first-author · 10 since 2021Human-computer interaction and ubiquitous computing · 4 · 1 first-author · 4 since 2021Graphics, computer vision, multimedia, augmented reality and games · 3 · 3 since 2021Artificial intelligence and machine learning · 1 · 1 since 2021Systems, architecture and hardware · 1
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2026 | Reliable Metal Foreign Object Detection for Mobile Wireless Charging via Harmonic FingerprintingabstractWireless charging eliminates cumbersome cables, revolutionizing how to charge mobile devices, yet reliably detecting metal foreign objects (e.g., keys, SIM ejectors, paper clips) poses a persistent challenge. This detection is critical, as such objects can inadvertently enter the charging zone, absorb energy, and trigger overheating, diminished efficiency, device damage, and even burns or fires. Existing approaches mainly monitor energy loss at the mobile device to infer intrusions, but this loss mixes inherent system dissipation with object-induced effects, both highly variable across devices and conditions, which often results in missed detections (as found in a range of commercial chargers). In this paper, we present Met-Sentry, a novel system design that takes a fundamentally different approach. Our key insight is that, beyond energy absorption, metal foreign objects also alter the electromagnetic field: they disproportionately attenuate the high-frequency harmonics in the in-band communication waveforms during charging, acting as a low-pass filter and yielding a distinctive, physics-grounded fingerprint of their presence. MetSentry captures and analyzes these fingerprints through a lightweight sensing circuit and a tailored software pipeline that extracts robust, discriminative features, which can be seamlessly integrated into wireless chargers, enabling reliable detection. Extensive experiments with various commercial wireless chargers, smartphones, and metal foreign objects demonstrate that MetSentry consistently outperforms both built-in charger detection and state-of-the-art methods. © 2026 Copyright held by the owner/author(s). Shenyao Jiang, Yang Liu 0101, Lixiang Han, Xinyu Wang 0030, Hao Zhou 0001, Zhenjiang Li 0001 |
MobiSys | 2 |
| 2026 | NeuroPath: Practically Adopting Motor Imagery Decoding through EEG SignalsabstractMotor Imagery (MI) is an emerging Brain–Computer Interface (BCI) paradigm in which a person imagines a body movement without any physical action. By decoding the scalp-recorded electroencephalography (EEG) signals, BCIs can establish direct communication pathways to control external devices, offering significant potential in prosthetics, rehabilitation, and human–computer interaction. However, existing solutions remain difficult to deploy in practice. (i) Most employ independent, opaque models for each MI task. This fragmented methodology lacks a unified architectural foundation. Consequently, these models are trained in isolation and fail to learn robust representations from diverse datasets, which often results in modest performance. (ii) They primarily adopt fixed sensor deployment, whereas real-world setups vary in electrode number and placement, causing models trained on one configuration to fail under another. (iii) Performance degrades sharply under low-SNR conditions typical of consumer-grade EEG. Together, these limitations hinder the practical adoption of MI-based BCIs. Jiani Cao, Kun Wang 0051, Yang Liu 0101, Zhenjiang Li 0001 |
SenSys | 3 |
| 2026 | NutriEar: Robust Nutrition-Aware Food Classification from In-Ear Acoustic SignalsabstractConvenient tracking of food intake is essential for linking diet to health, enabling personalised nutrition guidance, early metabolic risk detection, and prevention of chronic disease. Recent wearable sensing advances have begun to automate eating monitoring. However, these systems largely focus on detecting when users eat and only weakly address what they eat. In particular, state-of-the-art solutions typically cover only a narrow range of foods or textures and rely on strong assumptions about individual eating behaviour. Moreover, they overlook the nutritional implications most relevant to end users, limiting their usefulness for real-world dietary guidance. In this paper, we present NutriEar, an in-ear audio sensing system for nutrition-aware classification of food intake from chewing sounds. Rather than recognising arbitrary food types, NutriEar maps in-ear acoustics to an eight-class nutrition-texture taxonomy grounded in food science, capturing both dominant macronutrient role and mechanical texture. NutriEar records in-ear audio during eating, segments chewing events, and derives a hybrid representation combining engineered acoustic features with learned embeddings from supervised contrastive learning, enabling a compact nutrition-aware classification pipeline. Evaluation on a dataset collected from 15 users consuming over 30 food types under varied eating conditions shows that NutriEar achieves 80.18% average leave-one-subject-out (LOSO) accuracy and outperforms state-of-the-art baselines. These results highlight the untapped potential of earable audio sensing as a practical pathway toward everyday dietary monitoring with meaningful nutritional insights. Zoey Xiaochen Tan, Yang Liu 0101, Kayla-Jade Butkow, Cecilia Mascolo |
SenSys | 2 |
| 2026 | Short Paper: EarSleeve: Transforming Everyday Earphones into a 12-Lead ECG Sensing PlatformabstractAchieving multi-lead electrocardiography (ECG) in consumer-grade wearables remains challenging, as most devices provide only a few electrodes and cannot capture spatially diverse cardiac signals. Conventional 12-lead ECG, while clinically standard, requires ten electrodes across the body, confining its use to medical environments. We present EarSleeve, a modular dual-electrode eartip sleeve that transforms off-the-shelf earphones into a 12-lead ECG sensing platform through a human-in-the-loop design. Each sleeve embeds two conductive electrodes and electrically links both sides to form a four-electrode configuration. EarSleeve simultaneously records six limb leads and reconstructs 12-lead–equivalent ECG signals by sequentially contacting standard chest locations. In a 12-user study, EarSleeve captures clear ECG waveforms across all leads and is evaluated against an FDA-cleared reference. Results demonstrate the feasibility of reconstructing 12-lead–equivalent ECG signals using a minimum-electrode configuration under controlled conditions. To our knowledge, EarSleeve is the first system to achieve this with off-the-shelf earphones. Junxi Xia, Dogaç Eldenk, Yang Liu 0101, Stephen Xia |
SenSys | 4 |
| 2025 | SmarTeeth: Augmenting Manual Toothbrushing with In-ear Microphones
Qiang Yang 0018, Yang Liu 0101, Jake Stuchbury-Wass, Kayla-Jade Butkow, Emeli Panariti, Dong Ma 0001, Cecilia Mascolo |
CHI | 2 |
| 2025 | Cognitive Load Monitoring via Earable Acoustic SensingabstractThe rapid adoption of ear-worn devices (earables) has shown significant potential for continuous health monitoring. Despite their close proximity to the human brain and diverse sensing capabilities, the exploration of earable sensing in relation to cognitive function remains underexplored. Building on theoretical and empirical foundations regarding the interplay between cognitive load, auditory complexity, and changes in hearing characteristics influenced by brain function, this study is the first to leverage earable acoustic sensing to assess cognitive load. We specifically designed auditory tasks to elicit four levels of cognitive load and used otoacoustic emissions (OAEs) to measure cochlear response changes in response to cognitive load. By utilizing both audio content indicating auditory complexity and OAEs reflecting hearing characteristic changes, we designed machine learning pipelines to automate the assessment in a four-class cognitive detection task, achieving an accuracy of 68.88%. This research opens a new pathway for using earable acoustic sensing in monitoring cognitive function and holds great potential for future cognitive augmentation. Jiatao Quan, Khaldoon Al-Naimi, Xijia Wei, Yang Liu 0101, Fahim Kawsar, Alessandro Montanari, Ting Dang |
ICASSP | 4 |
| 2025 | Towards Detecting Auditory Attention from in-Ear Muscle Contractions using Commodity EarbudsabstractIn a world dominated by podcasts and audiobooks, maintaining auditory attention is essential, yet lapses in focus are common. Auditory attention is crucial for effective communication and comprehension in a distraction-filled environment, as it enables us to focus on important sounds while avoiding external distractions. This work introduces a novel, imperceptible method for detecting auditory attention using earbuds by monitoring muscle movement within the ear canal. We employ an ultrasound-based sensing technique to track phase changes in reflected signals, detecting muscle vibrations associated with shifts in attention. A preliminary user study reveals significant changes in in-ear signal characteristics when participants switch between auditory and cognitive tasks. We show that our system can classify periods of auditory attention and lack of it with an accuracy of 85.7% and a variance of 0.0033. Our findings pave the way for earables that continuously monitor and enhance auditory attention in real-time. Harshvardhan C. Takawale, Yang Liu 0101, Khaldoon Al-Naimi, Fahim Kawsar, Alessandro Montanari |
ICASSP | 2 |
| 2025 | RespEar: Earable-Based Robust Respiratory Rate MonitoringabstractContinuous respiratory rate (RR) monitoring is essential for understanding physical and mental health, as well as tracking fitness. However, performing reliable and non-obtrusive RR monitoring across diverse daily routines and activities is still an open research problem. In this work, we present RespEar, a pipeline for robust RR monitoring across various sedentary and active scenarios using earphones. RespEar relies solely on in-ear microphones, repurposing them for continuous RR monitoring purposes. Specifically, leveraging the unique properties of in-ear audio, RespEar enables the use of respiratory sinus arrhythmia (RSA) and locomotor respiratory coupling (LRC), physiological couplings between cardiovascular activity, gait and respiration, to determine the RR. This effectively addresses the challenges posed by the almost imperceptible breathing signals encountered during common daily activities. Additionally, RespEar uniquely identifies and addresses three key practical issues for the RSA and LRC-based solutions and introduces a suite of meticulously crafted signal processing techniques to enhance the accuracy of RR measurements. With data collected from 18 subjects over 8 activities, RespEar measures RR with a mean absolute error (MAE) of 1.48 breaths per minute (BPM) and a mean absolute percent error (MAPE) of 9.12% in sedentary conditions, and a MAE of 2.28 BPM and a MAPE of 11.04% in active conditions, respectively. To the best of our knowledge, RespEar is the first earable-based system capable of accurately determining RR in a variety of realistic settings. Yang Liu 0101, Kayla-Jade Butkow, Jake Stuchbury-Wass, Adam Pullin, Dong Ma 0001, Cecilia Mascolo |
PerCom | 1 |
| 2025 | WalkEar: Holistic Gait Monitoring using EarablesabstractGait behaviour is a key health metric. Temporal, spatial and kinetic walking gait parameters are valuable in enhancing sport performance and early health diagnostics Full gait assessment requires a gait clinic and existing wearable gait tracking systems typically measure isolated subsets of parameters tailored to specific applications. This is useful when the condition to be monitored is known, but fails to offer a comprehensive view of an individual’s gait traits when their pathology is unknown or changing, or a general assessment is required. To support holistic walking gait tracking, we introduce WalkEar, a novel sensing platform designed to simultaneously track gait parameters using commodity earbuds. WalkEar operates by detecting gait events to derive temporal gait parameters and segment the IMU data. WalkEar then progresses earable gait assessment by, for the first time, estimating kinetic gait parameters and reconstructing the vGRF curve using machine learning. Each parameter is calculated on a step-to-step basis for gait variability and asymmetry. We developed an earbud prototype and collected data from 13 participants using gold standard force plates and instrumented treadmill ground truth. Extensive experiments demonstrate the promising performance of WalkEar, achieving an overall MAPE of 5.1% in estimating gait, 2.0% MAPE on kinetic gait parameters, and an NRMSE of 5.3% for vGRF curve reconstruction. Jake Stuchbury-Wass, Yang Liu 0101, Kayla-Jade Butkow, Joshua Carter, Qiang Yang 0018, Mathias Ciliberto, Ezio Preatoni, Dong Ma 0001, Cecilia Mascolo |
PerCom | 2 |
| 2024 | Towards Enabling DPOAE Estimation on Single-Speaker EarbudsabstractDistortion Product OtoAcoustic Emissions (DPOAEs) represents faint cochlear responses to dual-frequency stimuli, commonly employed in hearing screening. This paper introduces an innovative approach to trigger DPOAEs using single-speaker earbuds. Due to their compact size, the speakers used in the earbuds exhibit nonlinear behavior, leading to Inter-Modulation Distortions (IMDs) that interfere with DPOAE signals. Conventional medical devices employ dual speakers to mitigate this distortion, such a solution is impractical for space-constrained earbuds. To address this challenge, we propose a method that triggers DPOAEs while circumventing IMDs by designing a stimulus signal that alternates between the two frequencies necessary for triggering DPOAEs. The performance of our system was evaluated through a preliminary user study involving 8 participants, and it demonstrated a median correlation of 0.65 when compared to a medical-grade reference device. Irtaza Shahid, Khaldoon Al-Naimi, Ting Dang, Yang Liu 0101, Fahim Kawsar, Alessandro Montanari |
ICASSP | 4 |
| 2024 | Towards Open Respiratory Acoustic Foundation Models: Pretraining and BenchmarkingabstractRespiratory audio, such as coughing and breathing sounds, has predictive power for a wide range of healthcare applications, yet is currently under-explored. The main problem for those applications arises from the difficulty in collecting large labeled task-specific data for model development. Generalizable respiratory acoustic foundation models pretrained with unlabeled data would offer appealing advantages and possibly unlock this impasse. However, given the safety-critical nature of healthcare applications, it is pivotal to also ensure openness and replicability for any proposed foundation model solution. To this end, we introduce OPERA, an OPEn Respiratory Acoustic foundation model pretraining and benchmarking system, as the first approach answering this need. We curate large-scale respiratory audio datasets ($\sim$136K samples, over 400 hours), pretrain three pioneering foundation models, and build a benchmark consisting of 19 downstream respiratory health tasks for evaluation. Our pretrained models demonstrate superior performance (against existing acoustic models pretrained with general audio on 16 out of 19 tasks) and generalizability (to unseen datasets and new respiratory audio modalities). This highlights the great promise of respiratory acoustic foundation models and encourages more studies using OPERA as an open resource to accelerate research on respiratory audio for health. The system is accessible from https://github.com/evelyn0414/OPERA. Yuwei Zhang 0001, Tong Xia, Jing Han 0010, Yu Wu 0021, Georgios Rizos, Yang Liu 0101, Mohammed Mosuily, Jagmohan Chauhan, Cecilia Mascolo |
NeurIPS | 6 |
| 2024 | An evaluation of heart rate monitoring with in-ear microphones under motionabstractWith the soaring adoption of in-ear wearables, the research community has started investigating suitable in-ear heart rate detection systems. Heart rate is a key physiological marker of cardiovascular health and physical fitness. Continuous and reliable heart rate monitoring with wearable devices has therefore gained increasing attention in recent years. Existing heart rate detection systems in wearables mainly rely on photoplethysmography (PPG) sensors, however, these are notorious for poor performance in the presence of human motion. In this work, leveraging the occlusion effect that enhances low-frequency bone-conducted sounds in the ear canal, we investigate for the first time in-ear audio-based motion-resilient heart rate monitoring. We first collected heart rate-induced sounds in the ear canal using an in-ear microphone under seven stationary activities and two full-body motion activities (i.e., walking, and running). Then, we devised a novel deep learning based motion artefact (MA) mitigation framework to denoise the in-ear audio signals, followed by a heart rate estimation algorithm to extract heart rate. With data collected from 15 subjects over nine activities, we demonstrate that hEARt, our end-to-end approach, achieves a mean absolute error (MAE) of 1.88 ± 2.89 BPM, 6.83 ± 5.05 BPM, and 13.19 ± 11.37 BPM for stationary, walking, and running, respectively, opening the door to a new non-invasive and affordable heart rate monitoring with usable performance for daily activities. Not only does hEARt outperform previous in-ear heart rate monitoring work, but it outperforms reported in-ear PPG performance. Kayla-Jade Butkow, Ting Dang, Andrea Ferlini, Dong Ma 0001, Yang Liu 0101, Cecilia Mascolo |
Pervasive Mob. Comput. | 5 |
| 2024 | Practical Gaze Tracking on Any Surface With Your PhoneabstractThis paper introduces ASGaze, a novel gaze tracking system using the RGB camera of smartphones. ASGaze improves the accuracy of existing methods and uniquely tracks gaze points on various surfaces, including phone screens, computer displays, and non-electronic surfaces like whiteboards or paper - a situation that is challenging for existing methods. To achieve this, we revisit the 3D geometric eye model, commonly used in high-end commercial trackers, and it has the potential to achieve our goals. To avoid the high cost of commercial solutions, we identify three fundamental issues when processing the eye model with an RGB camera, including how to accurately extract iris boundary that is the meta-information in our design, how to remove ambiguity from iris boundary to gaze point transformation, and how to map gaze points onto the target surface. Furthermore, as we consider deploying ASGaze in real-world applications, two additional challenges should be addressed: how to automatically and accurately annotate the training dataset to reduce manual labor and time costs, and how to accelerate the inference speed of ASGaze on mobile devices to improve user experience. We propose effective techniques to resolve these issues. Our prototype and experiments on three tracking surfaces demonstrate significant performance gains. Jiani Cao, Jiesong Chen, Chengdong Lin, Yang Liu 0101, Kun Wang 0051, Zhenjiang Li 0001 |
IEEE Trans. Mob. Comput. | 4 |
| 2024 | Finger Tracking Using Wrist-Worn EMG SensorsabstractThis paper introduces WETrak, a finger tracking system using wrist-worn electromyography (EMG) sensors. Recent finger tracking methods mainly employ EMGs on armbands. Compared to a range of contactless methods using cameras or wireless, they are not limited by high computational costs, privacy concerns, and mobility, while unlike other wearable-based approaches, they do not require the deployment of sensors on the user's hands. However, users need to wear an additional armband on their forearm each time solely for tracking purpose, which hinders the widespread adoption of finger tracking in practice. This paper investigates the feasibility of moving EMG sensors from the forearm to the wrist for finger tracking. WETrak inherits the advantages of existing EMG-based armband tracking while avoiding the limitation of requiring additional armbands, which brings a strong incentive for integrating EMG sensors into wrist-worn wearables in the future. As sensor placement varies, we find new challenges in determing good locations to place sensors to gather useful information to capture all finger movements and using low-quality signals to still ensure accurate tracking. In this paper, we introduce new, efficient solutions to these problems. We develop a prototype, and the results show that WETrak outperforms the state-of-the-art method and performs consistently well under various settings. Jiani Cao, Yang Liu 0101, Lixiang Han, Zhenjiang Li 0001 |
IEEE Trans. Mob. Comput. | 2 |
| 2024 | Enhancing the Applicability of Sign Language TranslationabstractThis paper addresses a significant problem in American Sign Language (ASL) translation systems that has been overlooked. Current designs collect excessive sensing data for each word and treat every sentence as new, requiring the collection of sensing data from scratch. This approach is time-consuming, taking hours to half a day to complete the data collection process for each user. As a result, it creates an unnecessary burden on end-users and hinders the widespread adoption of ASL systems. In this study, we identify the root cause of this issue and proposeGASLA–a wearable sensor-based solution that automatically generates sentence-level sensing data from word-level data. An acceleration approach is further proposed to optimize the data generation speed. Moreover, due to the gap between the generated sentence data and directly collected sentence data, a template strategy is proposed to make the generated sentences more similar to the collected sentence. The generated data can be used to train ASL systems effectively while reducing overhead costs significantly.GASLAoffers several benefits over current approaches: it reduces initial setup time and future new-sentence addition overhead; it requires only two samples per sentence compared to around ten samples in current systems; and it improves overall performance significantly. Jiao Li 0002, Jiakai Xu, Yang Liu 0101, Weitao Xu, Zhenjiang Li 0001 |
IEEE Trans. Mob. Comput. | 3 |
| 2023 | Keystroke Recognition With the Tapping Sound Recorded by Mobile Phone MicrophonesabstractMobile phones nowadays are equipped with at least dual microphones. We find when a user is typing on a phone, the sounds generated from the vibration caused by finger’s tapping on the screen surface can be captured by both microphones, and these recorded sounds alone are informative enough to localize the user’s keystrokes. This ability can be leveraged to enable useful application designs, while it also raises a crucial privacy risk that the private information typed by users on mobile phones has a great potential to be leaked through such a recognition ability. In this paper, we address two key design issues and demonstrate, more importantly alarm people, that this risk is possible, which could be related to many of us when we use our mobile phones. We implement our proposed techniques in a prototype system and conduct extensive experiments. The evaluation results indicate promising successful rates for more than 4000 keystrokes from different users on various types of mobile phones. Tao Chen 0033, Yang Liu 0101, Jiao Li 0002, Zhenjiang Li 0001 |
IEEE Trans. Mob. Comput. | 3 |
| 2022 | GASLA: Enhancing the Applicability of Sign Language TranslationabstractThis paper studies an important yet overlooked applicability issue in existing American sign language (ASL) translation systems. With excessive sensing data collected for each ASL word already, current designs treat every to-be-recognized sentence as new and collect their sensing data from scratch, while the amounts of sentences and the data samples per sentence are large usually. It takes a long time to complete the data collection for each single user, e.g., hours to a half day, which brings non-trivial burden to the end users inevitably and prevents the broader adoption of the ASL systems in practice. In this paper, we figure out the reason causing this issue. We present GASLA atop the wearable sensors to instrument our design. With GASLA, the sentence-level sensing data can be generated from the word-level data automatically, which can be then applied to train ASL systems. Moreover, GASLA has a clear interface to be integrated to existing ASL systems for overhead reduction directly. With this ability, sign language translation could become highly lightweight in both initial setup and future new-sentence addition. Compared with around 10 per-sentence data samples in current systems, GASLA requires 2–3 samples to achieve a similar performance. Jiao Li 0002, Yang Liu 0101, Weitao Xu, Zhenjiang Li 0001 |
INFOCOM | 2 |
| 2022 | Gaze Tracking on Any Surface with Your PhoneabstractThis paper introduces ASGaze, a new gaze tracking system designed using the common RGB camera from mobile phones. In addition to improving the accuracy of existing RGB camera-based gaze tracking methods, a novelty of ASGaze is that it can be configured to track gaze points on various surface areas commonly required in different applications, such as mobile phone screens, computer displays or even non-electronic surfaces like whiteboards or paper - a situation that is difficult for existing RGB camera-based methods to handle. To achieve the design of ASGaze, we revisit the 3D geometric model of the eye, which is widely adopted by high-end and commercial gaze trackers, and it has the potential to achieve our design goals. To avoid the high cost of commercial solutions, we identify three key issues to be addressed when processing the eye model with an RGB camera, including how to first accurately extract eye iris boundary that is the meta-information in our gaze tracking design, and then how to remove gaze ambiguity from iris boundary to gaze point transformation, and finally how to precisely map gaze points to the target tracking surface. In this paper, we propose a series of effective techniques to address these issues. We develop a prototype system and conduct extensive experiments on three different typical tracking surfaces to show promising performance gains compared to the recent solution. Jiani Cao, Chengdong Lin, Yang Liu 0101, Zhenjiang Li 0001 |
SenSys | 3 |
| 2020 | Mobile Phones Know Your Keystrokes through the Sounds from Finger's Tapping on the ScreenabstractMobile phones nowadays are equipped with at least dual microphones. We find when a user is typing on a phone, the sounds generated from the vibration caused by finger's tapping on the screen surface can be captured by both microphones, and these recorded sounds alone are informative enough to infer the user's keystrokes. This ability can be leveraged to enable useful application designs, while it also raises a crucial privacy risk that the private information typed by users on mobile phones has a great potential to be leaked through such a recognition ability. In this paper, we address two key design issues and demonstrate, more importantly alarm people, that this risk is possible, which could be related to many of us when we use our mobile phones. We implement our proposed techniques in a prototype system and conduct extensive experiments. The evaluation results indicate promising successful rates for more than 4000 keystrokes from different users on various types of mobile phones. Tao Chen 0033, Yang Liu 0101, Zhenjiang Li 0001 |
ICDCS | 3 |
| 2020 | Adversarial Attacks and Defenses on Cyber-Physical Systems: A SurveyabstractCyber-security issues on adversarial attacks are actively studied in the field of computer vision with the camera as the main sensor source to obtain the input image or video data. However, in modern cyber-physical systems (CPSs), many other types of sensors are becoming popularly used, such as surveillance sensors, microphones, and textual interfaces. A series of recent works investigates the adversarial attacks and the potential defenses in these noncamera sensor-based CPSs. Therefore, this article provides a systematic discussion on these existing works and serves as a complimentary summary of the adversarial attacks and defenses for CPSs beyond the field of computer vision. We first introduce a general working flow for adversarial attacks on CPSs. On this basis, a clear taxonomy is provided to organize existing attacks effectively and indicate where the defenses can be potentially performed in CPSs as well. Then, we discuss these existing attacks and defenses with detailed comparison studies. Finally, we point out concrete research opportunities to be further explored along this research direction. Jiao Li 0002, Yang Liu 0101, Tao Chen 0033, Zhenjiang Li 0001, Jianping Wang 0001 |
IEEE Internet Things J. | 2 |
| 2020 | aLeak: Context-Free Side-Channel from Your Smart Watch Leaks Your Typing PrivacyabstractWe revisit a crucial privacy problem in this paper - can the sensitive information, like the numeric passwords and personal data, frequently typed by user on mobile devices be inferred through the motion sensors of wearable device on user's wrist, e.g., smart watch or wrist band? Existing works have achieved the initial success under certain context-aware conditions, such as 1) the horizontal keypad plane, 2) the known keyboard size, and/or 3) the last keystroke on a fixed “enter” button. Taking one step further, the key contribution of this paper is to fully demonstrate, more importantly alarm people, the further risks of typing privacy leakage in much more generalized context-free scenarios, which are related to most of us for the daily usage of mobile devices. We validate this feasibility by addressing a series of unsolved challenges and developing a prototype system aLeak. Extensive experiments show the efficacy of aLeak, which achieves promising successful rates in the attack from more than 500 rounds of different users' typings on various mobile platforms without any context-related information. Yang Liu 0101, Zhenjiang Li 0001 |
IEEE Trans. Mob. Comput. | 1 |
| 2019 | When Wearable Sensing Meets Arm TrackingabstractIn this poster, we present our recent work, a wearable system for achieving real-time 3D arm skeleton. We have coped with the major challenge that the skeleton of each arm is determined from the locations of the elbow and wrist, whereas a wearable device only senses a single point from the wrist. Result shows that the potential solution space is huge. This underconstrained nature fundamentally challenges the achievement of accurate and real-time arm skeleton tracking. In this study, we propose Hidden Markov Model (HMM) state reorganization and hierarchical search two methods to improve the heavyweight computation of the state-of-art arm tracking model and achieve real-time tracking even on mobile phone. Yang Liu 0101, Chengdong Lin, Zhenjiang Li 0001, Zhidan Liu 0001, Kaishun Wu |
MobiSys | 1 |
| 2019 | Real-time Arm Skeleton Tracking and Gesture Inference Tolerant to Missing Wearable SensorsabstractThis paper presents ArmTroi, a wearable system for understanding and analyzing the detailed arm motions of people primarily by using the motion sensors from wrist-worn wearable devices. ArmTroi can achieve real-time 3D arm skeleton tracking and reliable gesture inference tolerant to missing wearable sensors for enabling numerous useful application designs. We have coped with two major challenges through ArmTroi. First, the skeleton of each arm is determined from the locations of the elbow and wrist, whereas a wearable device only senses a single point from the wrist. We find that the potential solution space is huge. This underconstrained nature fundamentally challenges the achievement of accurate and real-time arm skeleton tracking. Second, wearable sensors may not reliably provide sensory data. For example, devices are not worn by the user, yet the learning tools for gesture inference, such as deep learning, typically have static network structures, which require nontrivial network adaptation to match the input's varying availability and ensure reliable gesture inference. We propose effective techniques to address above challenges, and all computations can be conducted on the user's smartphone. ArmTroi is thus a fully lightweight and portable system. We develop a prototype and extensive evaluation shows the efficacy of the ArmTroi design. Yang Liu 0101, Zhenjiang Li 0001, Zhidan Liu 0001, Kaishun Wu |
MobiSys | 1 |
| 2019 | cDeepArch: A Compact Deep Neural Network Architecture for Mobile SensingabstractMobile sensing is a promising sensing paradigm in the era of Internet of Things (IoT) that utilizes mobile device sensors to collect sensory data about sensing targets and further applies learning techniques to recognize the sensed targets to correct classes or categories. Due to the recent great success of deep learning, an emerging trend is to adopt deep learning in this recognition process, while we find an overlooked yet crucial issue to be solved in this paper - The size of deep learning models should be sufficiently large for reliably classifying various types of recognition targets, while the achieved processing delay may fail to satisfy the stringent latency requirement from applications. If we blindly shrink the deep learning model for acceleration, the performance cannot be guaranteed. To cope with this challenge, this paper presents a compact deep neural network architecture, namely cDeepArch. The key idea of the cDeepArch design is to decompose the entire recognition task into two lightweight sub-problems: context recognition and the context-oriented target recognitions. This decomposition essentially utilizes the adequate storage to trade for the CPU and memory resource consumptions during execution. In addition, we further formulate the execution latency for decomposed deep learning models and propose a set of enhancement techniques, so that system performance and resource consumption can be quantitatively balanced. We implement a cDeepArch prototype system and conduct extensive experiments. The result shows that cDeepArch achieves excellent recognition performance and the execution latency is also lightweight. Tianzhang Xing, Yang Liu 0101, Zhenjiang Li 0001, Xiaoqing Gong, Xiaojiang Chen, Dingyi Fang |
IEEE/ACM Trans. Netw. | 3 |
| 2019 | Rulers on Our Arms: Waving to Measure Object Size through Contactless SensingabstractIn this article, we propose a mobile system, Aware , which turns our wearable or mobile device into a ruler. It can estimate the size of objects that could be large in size and not directly touchable by the user. Such a design will enable a rich set of applications that count on the size information of surrounding environments/objects. Aware purely utilizes the motion sensors on the device for object size measures. It can also integrate with the crowdsourcing feature for both performance improvement and result sharing. We propose a series of key techniques to address three major challenges in the Aware design: (1) user’s angle of line-of-sights to the object is used in the size measure but motion sensors track only the angle of arm’s waving, (2) motion sensors are noisy that require novel and effective data processing techniques, otherwise the errors could easily overwhelm the final result, and (3) in the crowdsourcing mode, Aware needs to identify vicinal objects of similar sizes and effectively fuse the measured sizes that correspond to the same object. We consolidate the above designs and implement Aware on Android platforms. Extensive experiments with four users show that Aware can achieve accurate measurement performance for the objects of various sizes in both indoor and outdoor environments. Yang Liu 0101, Yonghang Jiang, Zhenjiang Li 0001, Jianping Wang 0001 |
ACM Trans. Sens. Networks | 1 |
| 2018 | aLeak: Privacy Leakage through Context - Free Wearable Side-ChannelabstractWe revisit a crucial privacy problem in this paper - can the sensitive information, like the passwords and personal data, frequently typed by user on mobile devices be inferred through the motion sensors of wearable device on user's wrist, e.g., smart watch or wrist band? Existing works have achieved the initial success under certain context-aware conditions, such as 1) the horizontal keypad plane, 2) the known keyboard size, 3) and/or the last keystroke on a fixed “enter” button. Taking one step further, the key contribution of this paper is to fully demonstrate, more importantly alarm people, the further risks of typing privacy leakage in much more generalized context-free scenarios, which are related to most of us for the daily usage of mobile devices. We validate this feasibility by addressing a series of unsolved challenges and developing a prototype system aLeak. Extensive experiments show the efficacy of aLeak, which achieves promising successful rates in the attack from more than 300 rounds of different users' typings on various mobile platforms without any context-related information. Yang Liu 0101, Zhenjiang Li 0001 |
INFOCOM | 1 |
| 2018 | cDeepArch: A Compact Deep Neural Network Architecture for Mobile SensingabstractMobile sensing is a promising sensing paradigm that utilizes mobile device sensors to collect sensory data about sensing targets and further applies learning techniques to recognize the sensed targets to correct classes or categories. Due to the recent great success of deep learning, an emerging trend is to adopt deep learning in this recognition process, while we find an overlooked yet crucial issue to be solved in this paper - The size of deep learning models should be sufficiently large for reliably classifying various types of recognition targets, while the achieved processing delay may fail to satisfy the stringent latency requirement from applications. If we blindly shrink the deep learning model for acceleration, the performance cannot be guaranteed. To cope with this challenge, this paper presents a compact deep neural network architecture, namely cDeepArch. The key idea of the cDeepArch design is to decompose the entire recognition task into two lightweight sub-problems: context recognition and the context-oriented target recognitions. This decomposition essentially utilizes the adequate storage to trade for the CPU and memory resource consumptions during execution. In addition, we further formulate the execution latency for decomposed deep learning models and propose a set of enhancement techniques, so that system performance and resource consumption can be quantitatively balanced. We implement a cDeepArch prototype system and conduct extensive experiments. The result shows that cDeepArch achieves excellent recognition performance and the execution latency is also lightweight. Xiaoqing Gong, Yang Liu 0101, Zhenjiang Li 0001, Tianzhang Xing, Xiaojiang Chen, Dingyi Fang |
SECON | 3 |