In collaborative inference scenarios, semantic communication replaces raw data transmission by conveying task-oriented semantic features to improve bandwidth efficiency. However, under noisy wireless channels, the combined effects of semantic compression distortion and channel noise lead to severe information loss, resulting in degraded inference accuracy. To address this issue, this paper proposes a Multi-scale Semantic Compression Collaborative Inference (MSCCI) framework that achieves efficient, stable inference performance under high compression ratios and elevated noise levels. Specifically, a UNet-based encoder extracts multi-scale semantic features on an IoT device. These features are then integrated into a unified stream using a novel semantic fusion compression strategy, thereby substantially reducing communication overhead. The edge server decoder decompresses features and recovers image semantics via progressive upsampling and multi-scale semantic restoration. For noisy wireless channels, the framework incorporates Squeeze-and-Excite (SE) attention for dynamic feature channel weighting and residual connections for enhanced low-SNR robustness. Experimental results demonstrate that our collaborative inference framework for semantic communication outperforms state-of-the-art algorithms, and the approach’s effectiveness and robustness are verified across various channel conditions.

The dynamic mechanism of joint proactive caching and cache replacement, which involves placing content items close to cache-enabled edge devices ahead of time until they are requested, is a promising technique for enhancing traffic offloading and relieving heavy network loads. However, due to limited edge cache capacity and wireless transmission resources, accurately predicting users’ future requests and performing dynamic caching is crucial to effectively utilizing these limited resources. This paper investigates joint proactive caching and cache replacement strategies in a general mobile edge computing (MEC) network with multiple users under a cloud-edge-device collaboration architecture. The joint optimization problem is formulated as a markov decision process (MDP) problem with an infinite range of average network load costs, aiming to reduce network load traffic while efficiently utilizing the limited available transport resources. To address this issue, we design an Attention Weighted Deep Deterministic Policy Gradient (AWD2PG) model, which uses attention weights to allocate the number of channels from server to user, and applies deep deterministic policies on both user and server sides for Cache decision-making, so as to achieve the purpose of reducing network traffic load and improving network and cache resource utilization. We verify the convergence of the corresponding algorithms and demonstrate the effectiveness of the proposed AWD2PG strategy and benchmark in reducing network load and improving hit rate.

Past tongue-jaw movement interaction systems typically require dedicated hardware and are uncomfortable to use, limiting their scalability and generalizability. This paper introducesCanalScan, the first system that recognizes tongue-jaw movements using commodity speakers and microphones mounted on ubiquitous off-the-shelf devices (e.g., smartphones). What inspires us is that tongue-jaw movements always cause ear canal deformations, and we find that for different tongue-jaw movements, dynamic features of ear canal deformations present unique patterns on acoustic reflections in the ear canal. Specifically,CanalScanfirst sends an acoustic signal to the ear canal, then parses the reflection signals for tongue-jaw movements recognition. To eliminate the impacts of body movements, we develop a body movement noise filtering method and a dynamic segmentation method to identify and separate the tongue-jaw movements-associated ear canal deformations from other types of body movements. We further propose a sensor position detection method and a data transformation mechanism to reduce the impacts of diversities in-ear canal shapes and relative positions between sensors and the ear canal.CanalScanexplores twelve unique and consistent features and applies a random forest classifier to distinguish tongue-jaw movements. Extensive experiments with twenty participants validate the generalizability, effectiveness, robustness, and high accuracy ofCanalScan.

Recent literature advances motion sensors mounted on smartphones and AR/VR headsets to speech eavesdropping due to their sensitivity to subtle vibrations. The popularity of motion sensors in earphones has fueled a rise in their sampling rate, which enables various enhanced features. This paper investigates a new threat of eavesdropping via motion sensors of earphones by developing EarSpy, which builds on our observation that the earphone's accelerometer can capture bone conduction vibrations (BCVs) and ear canal dynamic motions (ECDMs) associated with speaking; they enable EarSpy to derive unique information about the wearer's speech. Leveraging a study on the motion sensor measurements captured from earphones, EarSpy gains abilities to disentangle the wearer's live speech from interference caused by body motions and vibrations generated when the earphone's speaker plays audio. To enable user-independent attacks, EarSpy involves novel efforts, including a trajectory instability reduction method to calibrate the waveform of ECDMs and a data augmentation method to enrich the diversity of BCVs. Moreover, EarSpy explores effective representations from BCVs and ECDMs, and develops a neural network model with character-level and word-level speech recognition models to realize speech recognition. Extensive experiments involving 14 participants demonstrate that EarSpy reaches a promising recognition for the wearer's speech.

The rapid advancement of mobile edge computing (MEC) networks has enabled the augmentation of the computational power of mobile devices (MDs) by offloading computationally intensive tasks to resource-rich edge nodes. This paper discusses the decision-making process for task offloading and resource allocation among multiple mobile devices connected to a base station. The primary objective is to minimize the time taken to complete tasks while simultaneously reducing energy consumption on the device under a time-varying wireless fading channel. This objective is formulated as an energy-efficiency cost (EEC) minimization problem, which cannot be solved by conventional methods. To address this challenge, we propose a dynamic offloading decision algorithm of dependent tasks (DODA-DT) that adjusts local task execution based on edge node status. The proposed algorithm facilitates fair competition among all devices for edge resources. Additionally, we use a deep reinforcement learning (DRL) algorithm based on an actor-critic learning structure to train the system to quickly identify near-optimal solutions. Numerical simulations demonstrate that the proposed algorithm effectively reduces the total cost of the task in comparison to previous algorithms.

Recent literature advances motion sensors mounted on smartphones and AR/VR headsets to speech eavesdropping due to their sensitivity to subtle vibrations. The popularity of motion sensors in earphones has fueled a rise in their sampling rate, which enables various enhanced features. This paper investigates a new threat of eavesdropping via motion sensors of earphones by developing EarSpy, which builds on our observation that the earphone’s accelerometer can capture bone conduction vibrations (BCVs) and ear canal dynamic motions (ECDMs) associated with speaking; they enable EarSpy to derive unique information about the wearer’s speech. Leveraging a study on the motion sensor measurements captured from earphones, EarSpy gains abilities to disentangle the wearer’s live speech from interference caused by body motions and vibrations generated when the earphone’s speaker plays audio. To enable user-independent attacks, EarSpy involves novel efforts, including a trajectory instability reduction method to calibrate the waveform of ECDMs and a data augmentation method to enrich the diversity of BCVs. Moreover, EarSpy explores effective representations from BCVs and ECDMs, and develops a convolutional neural model with Connectionist Temporal Classification (CTC) to realize accurate speech recognition. Extensive experiments involving 14 participants demonstrate that EarSpy reaches a promising recognition for the wearer’s speech.

Proper handwashing, having a crucial effect on reducing bacteria, serves as the cornerstone of hand hygiene. For elders with dementia, they suffer from a gradual loss of memory and difficulty coordinating handwashing steps. Proper assistance should be provided to them to ensure their hand hygiene adherence. Toward this end, we propose AWash, leveraging inertial measurement unit (IMU) readily available in most wrist-worn devices (e.g., smartwatches) to characterize handwashing actions and provide assistance. To monitor handwashing scenarios round-the-clock while achieving energy efficiency, we design methods that distinguish handwashing from other daily activities and dynamically adjust the sampling duty cycle. Upon detecting handwashing actions, we design several novel techniques to segment different handwashing actions and extract sensor-body inclination angles that handle particular interference of senile dementia patients. Moreover, a user-independent network model is built to recognize the handwashing actions of senile dementia patients without requiring their training data. Furthermore, we propose a transfer learning method that improves system performance. To meet users’ diverse needs, we use a state machine to make prompt decisions, supporting customized assistance. Extensive experiments on a prototype with eight older participants demonstrate that AWash can increase the user’s independence in the execution of handwashing.

With the rapid development of mobile devices and the fast increase of sensitive data, secure and convenient mobile authentication technologies are desired. Except for traditional passwords, many mobile devices have biometric-based authentication methods (e.g., fingerprint, voiceprint, and face recognition), but they are vulnerable to spoofing attacks. To solve this problem, we study new biometric features which are based on the dental occlusion and find that the bone-conducted sound of dental occlusion collected in binaural canals contains unique features of individual bones and teeth. Motivated by this, we propose a novel authentication system, TeethPass, which uses earbuds to collect occlusal sounds in binaural canals to achieve authentication. We design an event detection method based on spectrum variance and double thresholds to detect bone-conducted sounds. Then, we analyze the time-frequency domain of the sounds to filter out motion noises and extract unique features of users from three aspects: bone structure, occlusal location, and occlusal sound. Finally, we design an incremental learning-based Siamese network to construct the classifier. Through extensive experiments including 22 participants, the performance of TeethPass in different environments is verified. TeethPass achieves an accuracy of 96.8% and resists nearly 99% of spoofing attacks.

Mobile crowdsensing (MCS) enables a platform to recruit users to collectively perform sensing tasks from requesters. In order to maximize the completion qualities of tasks, an incentive mechanism should be well designed for the platform to incentivize high-quality users’ participation. The existing works largely adopt the Stackelberg game to model the strategic interactions in the incentive mechanism. However, there are practical issues that are less investigated in the context of the Stackelberg-based incentive mechanism. First, the platform has no knowledge about users’ sensing qualities beforehand due to their private information. Second, the platform needs users’ continuous participation in the long run, which results in fairness requirements. Third, it is also crucial to protect users’ privacy due to the potential privacy leakage concerns (e.g., sensing qualities) after completing tasks. In this article, we jointly address these issues and propose the three-stage Stackelberg-based incentive mechanism for the platform to recruit participants. In detail, we leverage combinatorial volatile multiarmed bandits (CVMABs) to elicit unknown users’ sensing qualities. We use the drift-plus-penalty (DPP) technique in Lyapunov optimization to handle the fairness requirements. We blur the quality feedback with tunable Laplacian noise such that the incentive mechanism protects locally differential privacy (LDP). Finally, we carry out experiments to evaluate our incentive mechanism. The numerical results show that our incentive mechanism achievessublinearregret performance to learn unknown quality with fairness and privacy guarantee.

Hand hygiene has a significant impact on human health. Proper handwashing, having a crucial effect on reducing bacteria, serves as the cornerstone of hand hygiene. For the elder with dementia, they suffer from a gradual loss of memory and difficulty in coordinating steps in the execution of handwashing. Proper assistance should be provided to them to ensure their hand hygiene adherence. Toward this end, we propose AWash, leveraging only commodity IMU sensor mounted on most wrist-worn devices (e.g., smartwatches) to characterize hand motions and provide assistance accordingly. To handle particular interference of senile dementia patients in IMU sensor readings, we design a number of effective techniques to segment handwashing actions, transform sensory input to body coordinate system, and extract sensor-body inclination angles. A hybrid neural network model is used to enable AWash to generalize to new users without retraining or adaptation, avoiding the trouble of collecting behavior information of every user. To meet the diverse needs of users with various executive functioning, we use a state machine to make prompt decisions, which supports customized assistance. Extensive experiments on a prototype with eight older participants demonstrate that AWash can increase the user's independence in the execution of handwashing.

Human-machine interface based on tongue-jaw movements has recently become one of the major technological trends. However, existing schemes have several limitations, such as requiring dedicated hardware and are usually uncomfortable to wear. This paper presents CanalScan, a nonintrusive system for tongue-jaw movement recognition using only commodity speaker and microphone mounted on ubiquitous off-the-shelf devices (e.g., smartphones). The basic idea is to send an acoustic signal, then captures its reflections and derive unique patterns of ear canal deformation caused by tongue-jaw movements. A dynamic segmentation method with Support Vector Domain Description is used to segment tongue-jaw movements. To combat sensor position-sensitive deficiency and ear-canal-shape-sensitive deficiency in multi-path reflections, we first design algorithms to assist users in adjusting the acoustic sensors to the same valid zone. Then we propose a data transformation mechanism to reduce the impacts of diversities in ear canal shapes and relative positions between sensors and the ear canal. CanalScan explores twelve unique and consistent features and applies a Random Forest classifier to distinguish tongue-jaw movements. Extensive experiments with twenty participants demonstrate that CanalScan achieves promising recognition for six tongue-jaw movements, is robust against various usage scenarios, and can be generalized to new users without retraining and adaptation.

Arterial blood pressure (ABP) monitoring using wearables has emerged as a promising approach to empower users with self-monitoring for effective diagnosis and control of hypertension. However, existing schemes mainly monitor ABP at discrete time intervals, involve some form of user effort, have insufficient accuracy, and require collecting sufficient training data for model development. To tackle these problems, we propose Crisp-BP, a novel ABP monitoring system leveraging the PPG sensor available in commercial wrist-worn devices (e.g., smartwatches or fitness trackers). It enables continuous, accurate, user-independent ABP monitoring and requires no behavior changes during collecting PPG data. The basic idea is to illuminate a skin/tissue, measure the light absorption, and characterize ABP-related blood volume change in the artery. To obtain accurate measurements and relieve the pain of training data collection, we use an arterial pulse extraction method that removes interference caused by capillary pulses. Moreover, we design a contact pressure estimation method to combat the deficiency of PPG waveform being sensitive to the contact pressure between the sensor and the skin. In addition, we leverage the great power of Bidirectional Long Short Term Memory and design a hybrid neural network model to enable user-independent ABP monitoring, so that users do not have to provide training data for model development. Furthermore, we propose a transfer learning method that first extracts general knowledge from online PPG data, then use it to improve the learning of a new model on our target problem. Extensive experiments with 35 participants demonstrate that Crisp-BP obtains the average estimation error of 0.86 mmHg and 1.67 mmHg and the standard deviation error of 6.55 mmHg and 7.31 mmHg for diastolic pressure and systolic pressure, respectively. These errors are within the acceptable range regulated by the FDA's AAMI protocol, which allows average errors of up to 5 mmHg and a standard deviation of up to 8 mmHg. Our results demonstrate that Crisp-BP is promising for improving the diagnosis and control of hypertension as it provides continuousness, comfort, convenience, and accuracy.

Mobile Crowd Sensing (MCS) enables the platform to offer data-based service by incentivizing mobile users to perform sensing task and collecting sensing data from them. Most of the existing works on MCS only consider designing incentive mechanisms for a single MCS platform. In this paper, we study the incentive mechanism in MCS with multiple platforms under two scenarios: competitive platform and cooperative platform. We correspondingly propose new competitive and cooperative mechanisms for each scenario. In the competitive platform scenario, platforms decide their prices on rewards to attract more participants, while the users choose which platform to work for. We model such a competitive platform scenario as a two-stage Stackelberg game. In the cooperative platform scenario, platforms cooperate to share sensing data with each other. We model it as many-to-many bargaining. Moreover, we first prove the NP-hardness of exact bargaining and then propose heuristic bargaining. Finally, numerical results show that (1) platforms in the competitive platform scenario can guarantee their payoff by optimally pricing on rewards and participants can select the best platform to contribute; (2) platforms in the cooperative platform scenario can further improve their payoff by bargaining with other platforms for cooperatively sharing collected sensing data.

Micro finger gesture recognition is an emerging approach to realize more friendly interaction between human and smart devices, especially for small wearable devices, such as smartwatches and virtual reality glasses. This paper proposes airFinger, a novel solution utilizing NIR light sensing to realize both real-time gesture recognition and finger tracking aiming at micro finger gestures. Using a custom NIR-based sensor with novel algorithms to capture subtle finger movements, airFinger enables to detect a rich set of micro finger gestures and track finger movements in terms of scrolling direction, velocity, and displacement. Besides, airFinger is capable of effective noise mitigation, gesture segmentation, and reducing false recognition due to the unintentional actions of users. Extensive experimental results demonstrate that airFinger has robustness against individual diversity, gesture inconsistency, and many other impacts. The overall performance reaches an average accuracy as high as 98.72% over a set of 8 micro finger gestures among 10, 000 gesture samples collected from 10 volunteers.

In this article, we propose points-of-interest (POI)-tagging App-assisted incentive mechanism (PTASIM), an incentive mechanism that explores the cooperation with POI-tagging App for mobile edge crowdsensing (MEC). PTASIM requests App to tag some edges to be POI, which further guides App users to perform tasks at that location. We further model the interactions of users, platform, and App by a three-stage decision process. App first determines the POI-tagging price to maximize its payoff. Platform and users subsequently decide how to determine tasks reward and select edges to be tagged, and how to select the best task to perform, respectively. We analyze the optimal solution in those stages. Specifically, we prove that greedy algorithm could provide the optimal solution for platform's payoff maximization in polynomial time. The numerical results show that: 1) the cooperation with App brings long-term and sufficient participation; and 2) the optimal strategies reduce platform's tasks cost as well as improve App's revenues.

Pedestrian dead reckoning enables pervasive indoor localization without a site survey on fingerprints or an intensive deployment of infrastructures. But accumulated errors in dead reckoning limit the spread of pervasive indoor location-based services. Existing landmark-based approaches mostly rely on resetting the user’s position with the landmark position only when the user is detected while on arrival at a landmark. However, such methods are still restricted by the specific movement patterns and sparse landmark distributions so that the opportunity for position calibration is limited. In this paper, an accurate peer-assisted localization system (calledSoundMark) on a smartphone with no prior infrastructure or fingerprinting is proposed. It calibrates mobile user’s dead reckoning position by leveraging the location constraints between another stationary user who arrives at a landmark. To detect whether a user arrives at a landmark, motion pattern is extracted by fusing the multiple sensors. Then, user activity in the landmark is decomposed to determine whether the user is stationary for performing audio ranging. Besides, SoundMark also applies a mobility-induced time-difference-of-arrival-based audio ranging to extract the location constraints between the peers for localization. SoundMark is implemented on the Android platform for evaluations. The results show that the accuracy of proposed peer-assisted localization is within 2.1 m at the percentage of 80%.

This paper explores the limits of acoustic ranging on smart phone in the scenario of device-free hand tracking. Tracking the hand is challenging since it requires continuously locating the moving hand in the air with fine resolution. Existing work on hand tracking relies on special hardware or requires users hold the mobile device. This paper presents EchoTrack, which continuously locates the hand by leveraging mobile audio hardware advances without special infrastructure supported. EchoTrack measures the distance from the hand to the speaker array embedded in smart phone via the chirp's Time of Flight (TOF). The speaker array and hand yield a unique triangle. The hand can be located with this triangular geometry. The trajectory accuracy can be improved with the method of Doppler shift compensation and trajectory correction (i.e., roughness penalty smoothing method). We implement a prototype on smart phone and the evaluation shows that EchoTrack can achieve tracking accuracy within about three centimeters of 76% and two centimeters of 48%.

Hand tracking systems are becoming increasingly popular as a fundamental HCI approach. The trajectory of moving hand can be estimated through smoothing the position coordinates collected from continuous localization. Therefore, hand localization is a key component of any hand tracking systems. This paper presents EchoLoc, which locates the human hand by leveraging the speaker array in Commercial Off-The-Shelf (COTS) devices (i.e., a smart phone plugged with a stereo speaker). EchoLoc measures the distance from the hand to the speaker array via the Time Of Flight (TOF) of the chirp. The speaker array and hand yield a unique triangle, therefore, the hand can be localized with triangular geometry. We prototype EchoLoc on iOS as an application, and find it is capable of localization with the average resolution within five centimeters of 73% and three centimeters of 48%.