Yuqin Yang, Haowu Zhou, Haoran Tu, Zhiwen Hui, Shiqi Yan, HaoYang Li, Dong She, Xianrong Yao, Yang Gao, Zhanpeng Jin. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Facial expressions are vital for effective communication, conveying emotions and health status. Traditional analysis methods, like manual annotations and geometric models, are labor-intensive and inadequate for complex situations. While vision-based approaches improve accuracy, they often struggle with environmental constraints and privacy concerns. Non-visual wearables offer flexibility but can be uncomfortable and power-hungry. To overcome these issues, we introduce IMUFace, an innovative earplug platform that uses inertial measurement units (IMUs) for real-time facial expression reconstruction. IMUFace captures facial motion data through IMUs in headphones and processes it with a deep learning model to estimate facial landmarks accurately. These predictions are then fitted to the FLAME model, creating realistic 3D facial animations. Compact and low-power, IMUFace represents a significant advancement in generating 3D facial animations for everyday use.