American Sign Language (ASL) is the primary language of many Deaf and Hard of Hearing (DHH) individuals. However, existing learning resources often lack timely, individualized feedback, leaving learners uncertain about signing accuracy. We introduce a novel egocentric ASL learning system that integrates stereo vision, error detection across four manual ASL parameters (handshape, orientation, location, movement), and large language model (LLM)–driven natural language feedback. To our knowledge, this is the first system to deliver error-aware, pedagogically grounded feedback for ASL learners. A formative study with 15 ASL teachers and 30 learners (both Deaf and hearing backgrounds) supports the motivation and design goals, while a system evaluation with 13 Deaf ASL participants (novice to advanced) practicing 230 signs provides initial evidence of system feasibility and short-term, pedagogically promising behavior within the primary user community. Across two complementary studies, we identify key design principles: prioritizing reliability over sensitivity, stratifying feedback by error severity, and leveraging egocentric alignment for natural practice. Collectively, these contributions establish a foundation for scalable ASL education and provide generalizable insights for designing AI-mediated feedback in Human-Computer Interaction (HCI).

With the proliferation of wearable devices and ubiquitous computing, context-aware music recommendation systems are evolving to deliver more personalized experiences. Traditional methods relying on explicit feedback, such as listening history and song ratings, struggle to adapt to users' dynamic contextual states, limiting their effectiveness. In this paper, we introduce Wear2Rec, a privacy-preserving, IoT-driven music recommendation system that leverages passive physiological, psychological, and environmental data from wearable devices to enhance personalization. At its core, Wear2Rec employs an innovative dual-expert-dual-task network architecture that separately extracts context and music features, minimizing cross-modal interference. Unlike conventional models, it simultaneously optimizes both music recommendation and mood improvement prediction, ensuring both relevant music suggestions and an emotionally supportive listening experience. Experimental results show its superiority, achieving 0.8411 AUC for music recommendation and 0.5928 MAE for mood prediction, outperforming traditional models by integrating emotional adaptation. Wear2Rec represents a significant step forward in human-centric, real-time recommendation systems, setting new standards for personalized music experiences in IoT-driven ubiquitous computing environments.

Facial expressions are vital for effective communication, conveying emotions and health status. Traditional analysis methods, like manual annotations and geometric models, are labor-intensive and inadequate for complex situations. While vision-based approaches improve accuracy, they often struggle with environmental constraints and privacy concerns. Non-visual wearables offer flexibility but can be uncomfortable and power-hungry. To overcome these issues, we introduce IMUFace, an innovative earplug platform that uses inertial measurement units (IMUs) for real-time facial expression reconstruction. IMUFace captures facial motion data through IMUs in headphones and processes it with a deep learning model to estimate facial landmarks accurately. These predictions are then fitted to the FLAME model, creating realistic 3D facial animations. Compact and low-power, IMUFace represents a significant advancement in generating 3D facial animations for everyday use.

Tidal volume (VT) is defined as the volume of inhaled and exhaled air during normal breath, which is crucial for maintaining respiratory function, such as adequate air exchange in and out of the body. However, existing estimation methods either require complex setups or involve inconvenient and expensive devices, such as spirometer and chestband. Thus, leveraging the advanced artificial intelligence (AI) and wearable devices, we aim to develop a novel, accessible and convenient approach to estimate tidal volume. In this study, we propose the VTMonitor system, which utilizes consumer earbuds’ motion sensor data to estimate the tidal volume. We conducted two experiments, collecting data either in lab or at home. After analyzing the data, our VTMonitor system is effective in measuring the tidal volume.

Wearable devices have become indispensable gadgets in people’s daily lives nowadays; especially wireless earphones have experienced unprecedented growth in recent years, which lead to increasing interest and explorations of user authentication techniques. Conventional user authentication methods embedded in wireless earphones that use microphones or other modalities are vulnerable to environmental factors, such as loud noises or occlusions. To address this limitation, we introduce EarPPG, a new biometric modality that takes advantage of the unique in-ear photoplethysmography (PPG) signals, altered by a user’s unique speaking behaviors. When the user is speaking, muscle movements cause changes in the blood vessel geometry, inducing unique PPG signal variations. As speaking behaviors and PPG signals are unique, the EarPPG combines both biometric traits and presents a secure and obscure authentication solution. The system first detects and segments EarPPG signals and proceeds to extract effective features to construct a user authentication model with the 1D ReGRU network. We conducted comprehensive real-world evaluations with 25 human participants and achieved 94.84% accuracy, 0.95 precision, recall, and f1-score, respectively. Moreover, considering the practical implications, we conducted several extensive in-the-wild experiments, including body motions, occlusions, lighting, and permanence. Overall outcomes of this study possess the potential to be embedded in future smart earable devices.

Sign language is a primary language used by deaf and hard-of-hearing (DHH) communities. However, existing sign language translation solutions primarily focus on recognizing manual markers. The non-manual markers, such as negative head shaking, question markers, and mouthing, are critical grammatical and semantic components of sign language for better usability and generalizability. Considering the significant role of non-manual markers, we propose the TransASL, a real-time, end-to-end system for sign language recognition and translation. TransASL extracts feature from both manual markers and non-manual markers via a customized eyeglasses-style wearable device with two parallel sensing modalities. Manual marker information is collected by two pairs of outward-facing microphones and speakers mounted to the legs of the eyeglasses. In contrast, non-manual marker information is acquired from a pair of inward-facing microphones and speakers connected to the eyeglasses. Both manual and non-manual marker features undergo a multi-modal, multi-channel fusion network and are eventually recognized as comprehensible ASL content. We evaluate the recognition performance of various sign language expressions at both the word and sentence levels. Given 80 frequently used ASL words and 40 meaningful sentences consisting of manual and non-manual markers, TransASL can achieve the WER of 8.3% and 7.1%, respectively. Our proposed work reveals a great potential for convenient ASL recognition in daily communications between ASL signers and hearing people.

With the aging of the population and the long-time wearing of earphones, hearing health has gradually emerged as a worldwide health issue. Early detection of hearing health conditions would greatly reduce potential risks with timely medical intervention. This study proposes an earphone-based ear condition monitoring system, named EarHealth, which is low-cost, non-invasive, and easily usable in daily life. It can detect three major hearing health conditions: ruptured eardrum, earwax buildup and blockage, and otitis media. By analyzing the recorded echoes evoked by a chirp sound stimulus, EarHealth recognizes the distinguishable characteristics from ear canal structure and eardrum mobility. EarHealth achieves an accuracy of 82.6% in 92 human subjects, including 27 normal subjects, 22 patients with ruptured eardrum, 25 patients with otitis media, and 18 patients with earwax blockage. EarHealth is the first earphone-based system capable of monitoring hearing health conditions by utilizing the ear canal geometry and eardrum mobility. It is anticipated that EarHealth would provide pervasive and proactive protection for hearing health.

To address the increasing challenges of counterfeit detection and IP protection for 3D printing, we propose that every 3D printer holds unique fingerprinting features characterized by the thermodynamic properties of the extruder hot-end and can be used as a new way of 3D watermarking. We prove that these physical fingerprints resulting from manufacturing imperfections and system variations exhibit distinct heating responses, namely “ThermoTag,” which can be represented as the distinguishable thermodynamic processes and, ultimately, the temperature readings during the preheating process. Experimental results show that, by only changing the hot-ends of the same model on the same 3D printer, we can achieve about 92% identification accuracy amongst 45 hot-ends. The permanence and robustness of ThermoTag for the same hot-end were examined, throughout a period of one month with hundreds of trials under different environmental temperature settings. Leveraging the hidden ThermoTag, an example of watermarking scheme in 3D printing is presented and evaluated.