With the popularization of face recognition technology in IoT-Cloud, the problem of privacy leakage caused by it is becoming more and more serious. Although traditional privacy protection schemes can improve security to a certain extent, there is still a risk of data leakage when facing semi-trusted cloud servers. To this end, this paper proposes an anonymized face verification scheme for IoT convergence scenarios, which achieves real-time retrieval and secure matching of dense face features in virtual device copies by combining homomorphic encryption (CKKS) and privacy information retrieval (PIR) for anonymized face verification. The scheme ensures that the semi-trusted cloud server cannot obtain user-specific index information and matching results. Experiments show that the scheme’s verification accuracy in the ciphertext state on the LFW dataset is consistent with the plaintext, up to 97.06%, and can complete a privacy-protected anonymized facial verification process within seconds. The scheme is feasible in security demanding scenarios.

With the increasing pervasiveness of smart earphones, it is appealing to propose more unobtrusive and convenient wearable authentication methods. Researchers have designed earphone-based authentication systems which utilize high-frequency audio signals to scan ear canal structure. Nevertheless, they possess shortcomings of low unobtrusiveness and robustness. In this article, we put forward an earphone-based passive authentication system which makes use of physiological and behavioral acoustic signals caused by a user’s natural actions, including putting on earphones and inner organs’ activities, respectively. By introducing attention mechanism into the network design, our method adaptively weighs two channel signals, and extracts stable fingerprints for different people, which relieves model retraining for unseen users and improves its scalability. We have built a real-time prototype called EarPrint by designing the earphones and a mobile application, and conducted comprehensive experiments under diverse settings. Experimental results demonstrate that EarPrint has low false acceptance rate (FAR) and equal error rate (EER) less than 1% and 5% in most cases, respectively.

User authentication and identification on smart devices has great significance in keeping data privacy and recommending personalized services. With the rising popularity of smart earphones recently, they open up a new world for users to enjoy music individually, but also bring about privacy concerns at the same time. Existing few research works propose positive sensing systems that emit and receive inaudible acoustic signals to authenticate users. However, they share shortcomings of intrusiveness to users, high power consumption, and purely focusing on authentication. Instead, in this paper, we propose a passive sensing system called${{\sf EarID}}$with low-cost customized earphones which attains user authentication and identification at once. It makes use of a embedded microphone to sense body sounds spread out through ear canals and extract ‘fingerprints’ as a novel biometric feature. With self-designed earphones, we design a deep learning-based real-time data processing pipeline and cope with external interference. Extensive experiments under different real-world settings show that${{\sf EarID}}$can achieve a rather low false acceptance rate of$3.4\%$for user authentication and a high$F1$score of$95.5\%$for legitimate user identification.

Emotions show high significance on human health. Automatic emotion recognition is helpful for monitoring psychological disorders, mental problems and exploring behavioral mechanisms. Existing approaches adopt costly and bulky specialized hardware such as EEG/ECG helmet, possess privacy risks, or with low accuracy and user experience. With the increasing popularity of wearables, people tend to equip multiple smart devices, which provides potential opportunity for emotion perception. In this paper, we present a pervasive and portable system called MW-Emotion to recognize common emotional states with multi-modal wearable devices. However, ubiquitous wearable devices perceive shallow information which is not obviously related to human emotions. MW-Emotion excavates intrinsic mapping relationship between emotions and sensing data. Our experiments show that MW-Emotion can recognize different emotion states with a relatively high accuracy of 83.1%.

User authentication and identification on smart devices has great significance in keeping data privacy and recommending personalized services. Existing few research works propose active sensing systems that emit and receive inaudible acoustic signals to authenticate users. But they share shortcomings of intrusiveness to users, high power consumption, and purely focusing on authentication. Instead, in this paper, we propose a passive sensing system called EarID with low-cost customized earpieces which attains user authentication and identification simultaneously. It makes use of a embedded microphone to sense body sounds spread out through ear canals and extract 'fingerprints' as a novel biometric feature. With self-designed earpieces, we design a deep learning-based real-time data processing pipeline. Extensive experiments under different real-world settings show that EarID can achieve a rather low false acceptance rate less than 5% for user authentication and a high F1 score of 96% for user identification.