Yuqin Yang, Haowu Zhou, Haoran Tu, Zhiwen Hui, Shiqi Yan, HaoYang Li, Dong She, Xianrong Yao, Yang Gao, Zhanpeng Jin. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

RageSense introduces a novel system for detecting and regulating player frustration during mobile gaming. Instead of relying on coarse emotion labels, RageSense estimates users’ valence and arousal levels in real time using near-ultrasonic acoustic sensing. By analyzing facial muscle movements via built-in smartphone speakers and microphones, our approach enables emotion sensing without requiring cameras or wearables, constituting a more unobtrusive, environment-resilient, and privacy-friendly approach than traditional emotion recognition. To transform detection into action, we integrate a large language model (LLM) that generates empathetic, context-aware interventions based on gameplay screenshots, behavioral signals, and emotional trajectories. These interventions are delivered in real time, tailored to the user’s emotional state, and designed to mitigate rage while enhancing player well-being. In a 53-participant field study, our system improved emotional state immediately after triggers and was preferred over random or template-based messages. To our knowledge, this is the first demonstration of near-ultrasonic, on-phone valence-arousal regression during mobile gameplay that directly drives real-time, context-aware interventions.

Silent speech interfaces (SSIs) promise private and noise-immune communication, but current solutions often sacrifice user comfort, mobility, or privacy. This paper introduces PPGSpeech, a novel SSI that overcomes these limitations by pioneering the use of photoplethysmography (PPG) acquired from a comfortable, necklace-style wearable device. Our core discovery is that subtle neck muscle movements during silent articulation induce distinct, measurable modulations in the underlying PPG signal. To harness this phenomenon, we developed a complete end-to-end system featuring (1) a custom neck-worn sensor for multi-wavelength PPG acquisition, (2) a deep learning pipeline that converts 1D PPG signals into 2D time-frequency images via Continuous Wavelet Transform (CWT) and classifies them using a lightweight CNN, and (3) a Pix2Pix GAN model to reconstruct audible speech from the captured signals. In a 16-participant study covering a vocabulary of 15 commands and four confounding actions, our user-dependent model achieved a recognition accuracy of 81.41% ± 9.74. Furthermore, our speech reconstruction achieved a Mean Opinion Score (MOS) of 3.48 and a Word Correct Rate (WCR) of 60.67%, demonstrating that the PPG signal is sufficiently rich to recover intelligible speech. By establishing the viability of neck-based PPG for silent speech, PPGSpeech offers a discreet, privacy-preserving, and continuously wearable paradigm for next-generation human-computer interaction.

Light detection and ranging (LiDAR) point cloud denoising is critical for reliable environmental perception in autonomous driving and robotics. To overcome the lack of real-noise datasets and the limited generalization of algorithms that rely on synthetic data, we construct a real-world LiDAR denoising dataset with noise-clean pairs, named RealLiD. Meanwhile, we propose a dual-heterogeneous-encoder network (DHE-Net) tailored for real-world noise. DHE-Net leverages spatial order information obtained from Knearest neighbor (KNN) sampling. It employs heterogeneous dual encoders to extract both the central semantic details and boundary distribution features of point cloud patches, thereby enabling more effective denoising. Experiments on RealLiD demonstrate that DHE-Net substantially outperforms mainstream denoising algorithms across multiple metrics, including chamfer distance, thereby proving its robustness and practicality under real-world noise conditions. The dataset and code will be released as open-source after publication to support future research.

With the proliferation of wearable devices and ubiquitous computing, context-aware music recommendation systems are evolving to deliver more personalized experiences. Traditional methods relying on explicit feedback, such as listening history and song ratings, struggle to adapt to users' dynamic contextual states, limiting their effectiveness. In this paper, we introduce Wear2Rec, a privacy-preserving, IoT-driven music recommendation system that leverages passive physiological, psychological, and environmental data from wearable devices to enhance personalization. At its core, Wear2Rec employs an innovative dual-expert-dual-task network architecture that separately extracts context and music features, minimizing cross-modal interference. Unlike conventional models, it simultaneously optimizes both music recommendation and mood improvement prediction, ensuring both relevant music suggestions and an emotionally supportive listening experience. Experimental results show its superiority, achieving 0.8411 AUC for music recommendation and 0.5928 MAE for mood prediction, outperforming traditional models by integrating emotional adaptation. Wear2Rec represents a significant step forward in human-centric, real-time recommendation systems, setting new standards for personalized music experiences in IoT-driven ubiquitous computing environments.

Facial expressions are vital for effective communication, conveying emotions and health status. Traditional analysis methods, like manual annotations and geometric models, are labor-intensive and inadequate for complex situations. While vision-based approaches improve accuracy, they often struggle with environmental constraints and privacy concerns. Non-visual wearables offer flexibility but can be uncomfortable and power-hungry. To overcome these issues, we introduce IMUFace, an innovative earplug platform that uses inertial measurement units (IMUs) for real-time facial expression reconstruction. IMUFace captures facial motion data through IMUs in headphones and processes it with a deep learning model to estimate facial landmarks accurately. These predictions are then fitted to the FLAME model, creating realistic 3D facial animations. Compact and low-power, IMUFace represents a significant advancement in generating 3D facial animations for everyday use.

Hand pose tracking is essential for advancing applications in human-computer interaction. Current approaches, such as vision-based systems and wearable devices, face limitations in portability, usability, and practicality. We present a novel wearable system that reconstructs 3D hand pose and estimates per-finger forces using a minimal ring-watch sensor setup. A ring worn on the finger integrates an inertial measurement unit (IMU) to capture finger motion, while a smartwatch-based single-channel electromyography (EMG) sensor on the wrist detects muscle activations. By leveraging the complementary strengths of motion sensing and muscle signals, our approach achieves accurate hand pose tracking and grip force estimation in a compact wearable form factor. We develop a dual-branch transformer network that fuses IMU and EMG data with cross-modal attention to predict finger joint positions and forces simultaneously. A custom loss function imposes kinematic constraints for smooth force variation and realistic force saturation. Evaluation with 20 participants performing daily object interaction gestures demonstrates an average Mean Per Joint Position Error (MPJPE) of 0.57 cm and a fingertip force estimation (RMSE: 0.213, r=0.76). We showcase our system in a real-time Unity application, enabling virtual hand interactions that respond to user-applied forces. This minimal, force-aware tracking system has broad implications for VR/AR, assistive prosthetics, and ergonomic monitoring.

Stress, a significant risk factor for chronic disease, manifests as changes in heart rate, respiration rate, and blood pressure. Non-invasive wearables like smartwatches can continuously track these physiological indicators to predict stress, enabling clinicians to develop and test interventions. However, most current devices are rigid and lack skin conformity, resulting in suboptimal signal quality and adherence during extended use. Furthermore, existing flexible sensors employ either electrocardiogram (ECG) or photoplethysmography (PPG), but not both, which is useful for calculating pulse arrival time (PAT) - known to correlate with stress. Addressing these challenges, we introduce HealthSense, a novel, flexible, and skin-conformable device that integrates ECG, PPG, and Inertial Measurement Unit (IMU) sensors into a single wearable. We assessed the comfort of wearing HealthSense and the feasibility of stress prediction by conducting a stress-induction study with 11 participants. Participants rated the comfort level of wearing the device on a Likert scale of 1-5, with 80% rating it as a 5 (most comfortable). Using statistical features, heart rate variability (HRV) related features, and PAT from our sensor data, we trained machine learning (ML) models to predict minute-level perceived and physiological stress with F1-scores of 85.5% and 87.7%, respectively. Additionally, using SHAP values, we identified PAT, systolic time, and pulse as the most significant contributors to the predictions. These findings enhance the understanding of physiological manifestations of stress and lays the groundwork for future stress-reduction interventions.

Wearable devices have become indispensable gadgets in people’s daily lives nowadays; especially wireless earphones have experienced unprecedented growth in recent years, which lead to increasing interest and explorations of user authentication techniques. Conventional user authentication methods embedded in wireless earphones that use microphones or other modalities are vulnerable to environmental factors, such as loud noises or occlusions. To address this limitation, we introduce EarPPG, a new biometric modality that takes advantage of the unique in-ear photoplethysmography (PPG) signals, altered by a user’s unique speaking behaviors. When the user is speaking, muscle movements cause changes in the blood vessel geometry, inducing unique PPG signal variations. As speaking behaviors and PPG signals are unique, the EarPPG combines both biometric traits and presents a secure and obscure authentication solution. The system first detects and segments EarPPG signals and proceeds to extract effective features to construct a user authentication model with the 1D ReGRU network. We conducted comprehensive real-world evaluations with 25 human participants and achieved 94.84% accuracy, 0.95 precision, recall, and f1-score, respectively. Moreover, considering the practical implications, we conducted several extensive in-the-wild experiments, including body motions, occlusions, lighting, and permanence. Overall outcomes of this study possess the potential to be embedded in future smart earable devices.

Sign language is a primary language used by deaf and hard-of-hearing (DHH) communities. However, existing sign language translation solutions primarily focus on recognizing manual markers. The non-manual markers, such as negative head shaking, question markers, and mouthing, are critical grammatical and semantic components of sign language for better usability and generalizability. Considering the significant role of non-manual markers, we propose the TransASL, a real-time, end-to-end system for sign language recognition and translation. TransASL extracts feature from both manual markers and non-manual markers via a customized eyeglasses-style wearable device with two parallel sensing modalities. Manual marker information is collected by two pairs of outward-facing microphones and speakers mounted to the legs of the eyeglasses. In contrast, non-manual marker information is acquired from a pair of inward-facing microphones and speakers connected to the eyeglasses. Both manual and non-manual marker features undergo a multi-modal, multi-channel fusion network and are eventually recognized as comprehensible ASL content. We evaluate the recognition performance of various sign language expressions at both the word and sentence levels. Given 80 frequently used ASL words and 40 meaningful sentences consisting of manual and non-manual markers, TransASL can achieve the WER of 8.3% and 7.1%, respectively. Our proposed work reveals a great potential for convenient ASL recognition in daily communications between ASL signers and hearing people.

Automated detection of intake gestures with wearable sensors has been a critical area of research for advancing our understanding and ability to intervene in people's eating behavior. Numerous algorithms have been developed and evaluated in terms of accuracy. However, ensuring the system is not only accurate in making predictions but also efficient in doing so is critical for real-world deployment. Despite the growing research on accurate detection of intake gestures using wearables, many of these algorithms are often energy inefficient, impeding on-device deployment for continuous and real-time monitoring of diet. This article presents a template-based optimized multicenter classifier that enables accurate intake gesture detection while maintaining low-inference time and energy consumption using a wrist-worn accelerometer and gyroscope. We designed an Intake Gesture Counter smartphone application (CountING) and validated the practicality of our algorithm against seven state-of-the-art approaches on three public datasets (In-lab FIC, Clemson, and OREBA). Compared with other methods, we achieved optimal accuracy (81.60% F1 score) and very low inference time (15.97 msec per 2.20-sec data sample) on the Clemson dataset, and among the top performing algorithms, we achieve comparable accuracy (83.0% F1 score compared with 85.6% in the top performing algorithm) but superior inference time (13.8x faster, 33.14 msec per 2.20-sec data sample) on the In-lab FIC dataset and comparable accuracy (83.40% F1 score compared with 88.10% in the top-performing algorithm) but superior inference time (33.9x faster, 16.71 msec inference time per 2.20-sec data sample) on the OREBA dataset. On average, our approach achieved a 25-hour battery lifetime (44% to 52% improvement over state-of-the-art approaches) when tested on a commercial smartwatch for continuous real-time detection. Our approach demonstrates an effective and efficient method, enabling real-time intake gesture detection using wrist-worn devices in longitudinal studies.

With the aging of the population and the long-time wearing of earphones, hearing health has gradually emerged as a worldwide health issue. Early detection of hearing health conditions would greatly reduce potential risks with timely medical intervention. This study proposes an earphone-based ear condition monitoring system, named EarHealth, which is low-cost, non-invasive, and easily usable in daily life. It can detect three major hearing health conditions: ruptured eardrum, earwax buildup and blockage, and otitis media. By analyzing the recorded echoes evoked by a chirp sound stimulus, EarHealth recognizes the distinguishable characteristics from ear canal structure and eardrum mobility. EarHealth achieves an accuracy of 82.6% in 92 human subjects, including 27 normal subjects, 22 patients with ruptured eardrum, 25 patients with otitis media, and 18 patients with earwax blockage. EarHealth is the first earphone-based system capable of monitoring hearing health conditions by utilizing the ear canal geometry and eardrum mobility. It is anticipated that EarHealth would provide pervasive and proactive protection for hearing health.

The original article can be found online.

To address the increasing challenges of counterfeit detection and IP protection for 3D printing, we propose that every 3D printer holds unique fingerprinting features characterized by the thermodynamic properties of the extruder hot-end and can be used as a new way of 3D watermarking. We prove that these physical fingerprints resulting from manufacturing imperfections and system variations exhibit distinct heating responses, namely “ThermoTag,” which can be represented as the distinguishable thermodynamic processes and, ultimately, the temperature readings during the preheating process. Experimental results show that, by only changing the hot-ends of the same model on the same 3D printer, we can achieve about 92% identification accuracy amongst 45 hot-ends. The permanence and robustness of ThermoTag for the same hot-end were examined, throughout a period of one month with hundreds of trials under different environmental temperature settings. Leveraging the hidden ThermoTag, an example of watermarking scheme in 3D printing is presented and evaluated.

Inspired by the biological nervous system and leveraging the recent advances in neuroscience, artificial neural networks (ANNs) have been extensively investigated and achieved great success in various domains. Nevertheless, the link between the intricate cognitive activity of the biological brain and the learning scheme of ANNs is still unclear and under-explored. Therefore, in this study we aim to preliminarily examine the association between these two parts and provide some explanations and interpretations on the memory-related characteristics associated with neural network topologies and internal connections by modeling the EEG/ERP brain activities with echo state network (ESN)-like architecture. Vector autoregressive (VAR) is adopted for parameter training. The experimental results partially verify the role of network connection pattern and synaptic strength in the memory representation of ANNs.

With robots and unmanned aerial vehicles (UAVs) being more and more employed in real-life scenarios for monitoring and surveillance, there is a increasing demand for deploying various video processing applications in mobile systems. However, with limited on-board computational resources and power consumption, the application in this domain requires that the tracking platforms equipped should have outstanding computing power to handle the tasks in real-time with high-accuracy, while at the same time, fit the highly constrained environment of small size, light weight, and low power consumption (SWaP) for the purpose of long-term surveillance. In this paper, we proposed a new autonomous object tracking system based on an embedded platform, leveraging the emerging neural network hardware which is capable of massive parallel pattern recognition processing and demands only a low level power consumption. Further, a prototype of the tracking system that combines a low-power neural network chip, CogniMem, and an embedded development board, BeagleBone, is developed. Our experimental results show that the power consumption for the entire system is only about 2. 25W, which signifies a promising future of applying ultra-low-power neuromorphic hardware as a accelerator in recognition tasks.

Object tracking which refers to automatic estimation of the trajectory is a challenging problem. To track the object robustly and efficiently, we explored an autonomous object tracking methodological framework that adopts the deep learning architectures, specifically the convolutional neural network (CNN) and the stacked denoising autoencoder (SDAE), as opposed to the most frequently used tracking algorithms that only learn the appearance of the tracked object. Moreover, we conduct a comparative study of both approaches in terms of tracking accuracy and efficiency. The results show that the features learned by both CNN and SDAE are very supportive in object tracking problem and the detailed comparisons are demonstrated in this work.