Advanced image generative models have led to concerns about malicious use, underscoring the necessity for generalizable detection methods. However, existing approaches tend to overfit to domain-specific forgery patterns, while overlooking complementary cues from different domains. Therefore, we introduce DySy-Det (Dynamic Synergy Detector), a novel framework that mines collaborative and robust forgery artifacts from multiple evidence domains. First, DySy-Det fine-tunes a CLIP vision transformer to extract high-level semantics for identifying conceptual inconsistencies, while generating attention maps that pinpoint key discriminative regions. Then, this semantic guidance, in the form of a mask, directs a targeted reconstruction process. By focusing on these salient areas, our approach effectively extracts localized reconstruction errors, thereby filtering out irrelevant background noise. Furthermore, inspired by the intrinsic generative mechanics of diffusion models, we introduce the concept of Reconstruction-Path Consistency (RPC), which quantifies the temporal stability of the denoising trajectory to expose dynamic generative artifacts. We capture this by computing noise alignment scores across multiple timesteps and encode them via a lightweight network. Extensive evaluations on GenImage and UniversalFakeDetect benchmarks demonstrate that DySy-Det outperforms the state-of-the-art detector by 6.14% and 1.57% in mean accuracy, respectively.

While text embeddings enable efficient semantic processing in LLMs, they remain vulnerable to inversion attacks that reconstruct sensitive original text. However, current defense methods typically treat text embeddings from the feature level independently, ignoring the exploitation of the mutual relation among the embedding construction pipeline. To address this limitation, we propose Eguard, a framework that effectively disrupts chains of relationships between the original semantic space and defended functional space. Our improvements manifest at two levels, i.e., the global-level and local-level mutual information. At the global level, we propose to minimize the statistical dependency between protected embeddings and their original inputs, effectively decoupling sensitive content from the semantic space accessible to adversaries. At the local level, we apply keyword-antonym contrastive learning to enforce semantic discriminability within the space of downstream utility. This synergy of global privacy control and local semantic alignment allows Eguard to achieve a superior privacy-utility trade-off than traditional defenses. Our approach significantly reduces privacy risks, protecting over 95 percent of tokens from inversion while maintaining high performance across downstream tasks consistent with original embeddings.

The integration of microphones into a broad array of devices, from consumer electronics to industrial sensors, introduces vulnerabilities to out-of-band injection attacks, including ultrasound, laser, electromagnetic, and magnetic field attacks. These attacks enable adversaries to inject inaudible or imperceptible commands, compromising systems without direct physical access. This paper presents a robust, passive detection framework designed to address the full spectrum of out-of-band attacks on microphone-equipped devices. Unlike prior approaches, our system leverages advanced speech disentanglement to separate semantic and acoustic features from recorded audio, enabling a refined analysis of injection artifacts within each feature domain. By quantifying entropy-based chaos within the disen tangled representations, we detect subtle spectral and structural irregularities indicative of injected signals. The system further incorporates a preliminary stage to identify carrier traces where applicable, expediting detection in cases such as ultrasound and laser attacks. Extensive evaluations across various device types, including smartphones, tablets, and microphones, demonstrate the system's high accuracy and stability, achieving an AUC of 98% under diverse conditions and attack configurations.

Multimodal language models (LMs) have shown significant potential for applications across various domains but remain vulnerable to adversarial attacks. Current research in white-box or black-box settings generally struggles with unrealistic attack assumptions and limited efficacy of targeted attacks. This paper introduces CoGA, a novel gray-box collaborative adversarial attack method for multimodal LMs. Under our gray-box settings, attackers have access only to the victim model’s input encoders. With the guidance of different modalities, we perturb the embedding representations from encoders to disrupt the semantic alignment across modalities, ultimately causing inaccurate outputs on various downstream tasks. Specifically, we integrate text embeddings into the loss calculations of the image attack and utilize image embeddings to guide the ranking of vulnerable words and the selection of final samples. Extensive experiments demonstrate that our method achieves superior attack performance across diverse models and tasks, suggesting the shared vulnerability of multimodal LMs in confronting adversarial challenges. Our work provides new insights into the security of multimodal LMs, facilitating the deployment of more robust and secure models in practical applications.

This work asks: with abundant, unlabeled real faces, how to learn a robust and transferable facial representation that boosts various face security tasks with respect to generalization performance? We make the first attempt and propose a self-supervised pretraining framework to learn fundamental representations of real face images, FSFM, that leverages the synergy between masked image modeling (MIM) and instance discrimination (ID). We explore various facial masking strategies for MIM and present a simple yet powerful CRFR-P masking, which explicitly forces the model to capture meaningful intra-region Consistency and challenging inter-region Coherency. Furthermore, we devise an ID network that naturally couples with MIM to establish underlying local-to-global Correspondence through tailored self-distillation. These three learning objectives, namely 3C, empower encoding both local features and global semantics of real faces. After pretraining, a vanilla ViT serves as a universal vision Foundation Model for downstream Face Security tasks: cross-dataset deepfake detection, cross-domain face anti-spoofing, and unseen diffusion facial forgery detection. Extensive experiments on 10 public datasets demonstrate that our model transfers better than supervised pretraining, visual and facial self-supervised learning arts, and even outperforms task-specialized SOTA methods.

Emerging live immersive viewing systems require streaming large 360 videos to users via limited wireless bandwidth. Neural-enhanced video streaming offers a promising solution by streaming down-scaled videos and enhancing them by client computation. However, prior systems treated video downscaling and enhancement separately, overlaying existing enhancement techniques onto current video infrastructure to accommodate legacy downscaling methods. This supplemental client design has led to spatial information loss and prohibitive model overheads in 360 video streaming systems. This paper presents Orbis, a redesigned, holistic neural-enhanced video streaming framework that integrates complementary down-scaling and enhancement for live immersive viewing. Orbis is empowered by an enhancement-driven interleaved downscaling approach, an inpainting-based enhancement model tailored to interleaved data, and a multi-scale tile adaptation scheme that optimizes immersive viewing experience in dynamic environments. Experimental results show that Orbis improves viewing experience by up to 60% and reduces wireless bandwidth by up to 49% compared to the best-performing baseline.

The widespread smart devices raise people’s concerns of being eavesdropped on. To enhance voice privacy, recent studies exploit the nonlinearity in microphone to jam audio recorders with inaudible ultrasound. However, existing solutions solely rely on energetic masking. Their simple-form noise leads to several problems, such as high energy requirements and being easily removed by speech enhancement techniques. Besides, most of these solutions do not support authorized recording, which restricts their usage scenarios. In this paper, we design an efficient yet robust system that can jam microphones while preserving authorized recording. Specifically, we propose a novel phoneme-based noise with the idea of informational masking, which can distract both machines and humans and is resistant to denoising techniques. Besides, we optimize the noise transmission strategy for broader coverage and implement a hardware prototype of our system. Experimental results show that our system can reduce the recognition accuracy of recordings to below 50% under all tested speech recognition systems, which is much better than existing solutions.

Pseudonyms used in Online social networks (OSNs) post a great challenge to fighting against cyber crimes. To build a strong case, law enforcement may want to link multiple user accounts with pseudonyms to a physical suspect. Images based camera fingerprinting has been used for account linking when a suspect takes pictures and videos with his camera and posts them online. However, image post-processing software may introduce noise into images. This noise is hard to eliminate by conventional strategies, is partly resident in the estimated photo-response non-uniformity (PRNU) fingerprints, and interferes with matching fingerprints. We define this noise as software noise, which pollutes PRNU fingerprints and affects accounts linking in online social networks. In this article, we propose new approaches for camera fingerprint matching given software noise. The key idea is to determine the PRNU hardware noise correlation component with our new test statistic–fingerprinttosoftware noise ratio (FITS). We performed extensive experiments and 10,000+ images taken by 90+ smartphones were used to validate our robust camera fingerprint matching system. The experiment results show FITS outperforms the state-of-the-art approaches for polluted fingerprints. This is the first work studying camera fingerprint matching with the presence of software noise.

Physical adversarial examples (AEs) have become an increasing threat to deploying deep neural network (DNN) models in the real world. Popular approaches adopt sticking-based or projecting-based strategies that stick the printed adversarial patches to objects or directly project the AE onto objects. Although effective, these methods require access to target objects and generate visible artifacts, which reduces the attack's stealthiness. In this article, we propose MagShadow, a new attack vector that leverages imperceptible electromagnetic (EM) signals to realize physical AEs. MagShadow utilizes the CCD camera sensor's susceptibility to EM injection attacks and induces fine-grained adversarial perturbations on the camera's captured image by injecting carefully-crafted signals with a low-cost portable device. As MagShadow directly manipulates the image sensor's output with EM signals, the attack requires no access to the target object and can keep stealthy. We study the feasibility of MagShadow in two typical DNN application scenarios (image classification and object detection) and design a framework to implement four different types of attacks, i.e., untargeted, targeted, hiding, and appearing attacks. Extensive real-world experiments on five different cameras are conducted, which demonstrate MagShadow's effectiveness against different popular DNN models (Inception v3, ResNet101, YOLO v3/v4).

Urban region profiling is essential for forecasting and decision-making in dynamic and noisy urban environments. However, existing approaches struggle with adversarial attacks, data incompleteness, and security vulnerabilities, which undermine predictive accuracy and reliability. This paper introduces Enhanced Urban Region Profiling with Adversarial Self-Supervised Learning (EUPAS), a robust framework that integrates adversarial contrastive learning with self-supervised and supervised objectives. To fortify resilience against adversarial attacks and noisy data, we introduce perturbation augmentation, a trickster generator, and a deviation copy generator, which collectively enhance the robustness of learned embeddings. EUPAS significantly outperforms state-of-the-art models in forecasting tasks, including crime prediction, check-in prediction, and land use classification, achieving up to 12.2% improvement in forecasting performance. Additionally, our model demonstrates superior resilience against transfer-based black-box and white-box attacks compared to baseline models. By addressing key security challenges in data-driven urban modeling, EUPAS provides a scalable and adversarially robust solution for smart city applications.

Photoplethysmography (PPG) has been extensively employed in commercial and medical products to assess human cardiac activities. However, despite PPG’s active role in improving people’s daily lives, research on the vulnerabilities of PPG systems is still in its infancy. This paper investigates the feasibility of deceiving PPG sensors in the physical domain. We propose FakePPG, which utilizes a low-cost Liquid Crystal Modulator (LCM) device to mimic the PPG signals of a legitimate user, thus deceiving both the PPG-based health assessment and potential authentication applications. To implement FakePPG in practical scenarios, we build the attack prototype using commercial off-the-shelf electronic components and further design an automated optimization and attack framework. By leveraging the modified multi-Gaussian model for parameterization, the evolutionary strategy for optimization, and the reference heart rate model for heartbeat variability alignment, FakePPG can achieve efficient, flexible, and automated PPG forgery against arbitrary users and heart states. Extensive experimental results show that FakePPG can achieve a success rate of 96.7% for Atrial Fibrillation (AFib) spoofing and 91.2% for identity spoofing, respectively, revealing a realistic threat to PPG systems.

Adversarial patch attacks pose a significant threat to deep learning models in real-world applications, such as autonomous driving, due to their physical feasibility and ease of deployment. Although several defenses exist, they often have limitations, such as requiring prior knowledge of adversarial patches, being tied to specific model architectures, or struggling to balance robustness with accuracy on clean inputs. In this work, we propose a novel Mask-Reconstruction-assisted Adversarial Training (MRAT) framework to enhance model robustness against adversarial patches while preserving accuracy. To achieve this, we first develop an adversarial example generation method that applies masks comprising randomly positioned square blocks with adversarial perturbations to clean images. This technique generates diverse adversarial examples for training, preventing the model from overfitting to a specific type of adversarial patch. To maintain accuracy on clean inputs while enhancing adversarial robustness, we incorporate a mask-reconstruction task into standard adversarial training paradigms. This task utilizes feature representations extracted from the backbone to reconstruct clean versions of adversarial examples. By jointly optimizing the reconstruction network and the classification model, the feature representations of adversarial examples retain essential features of their original images, leading to robust feature extraction and high classification accuracy. Extensive experimental results in both digital and physical domains demonstrate that MRAT significantly enhances the model’s defensive performance against adversarial patch attacks while maintaining high accuracy on clean inputs. Our code is available at: https://github.com/wanggroupAI/AI_Security/tree/main/Adversarial%20Example/MRAT.

With the increasing accessibility of drones, they have been warmly embraced across various sectors, especially in low-altitude logistics transportation. However, during drone delivery, legal drones dispatched by logistics companies are susceptible to malicious attacks, resulting in package theft or substitution. To address this, existing works focus on designing drone authentication to secure drone delivery. However, most of these methods require expensive specialized equipment, such as high-quality microphones and professional recording devices, resulting in high real-world application costs. In this paper, we propose DroneAudioID, a lightweight acoustic fingerprint-based drone authentication system that relies solely on common mobile devices. The basic idea is to employ acoustic fingerprints to authenticate different drones of the same model based on differences in fundamental frequency and harmonic components of drone audio. Specifically, the drone audio is recorded by a mobile device instead of sophisticated equipment. We apply wavelet transform to remove high-frequency noise during data preprocessing. Then, specialized filter banks are designed for feature extraction, leveraging the frequency characteristics of drone audio. Finally, we construct a Bi-Long Short-Term Memory (Bi-LSTM) with an Open-Max model for open-set classification. Extensive experiments are conducted on eight crafts drones of$DJI Mini2$, showing an authentication accuracy of 99.6%. A series of comprehensive experiments further validate DroneAudioID’s capability to defend against various attacks.

Deepfake technology has given rise to a spectrum of novel and compelling applications. Unfortunately, the widespread proliferation of high-fidelity fake videos has led to pervasive confusion and deception, shattering our faith that seeing is believing. One aspect that has been overlooked so far is that current deepfake detection approaches may easily fall into the trap of overfitting, focusing only on forgery clues within one or a few local regions. Moreover, existing works heavily rely on neural networks to extract forgery features, lacking theoretical constraints guaranteeing that sufficient forgery clues are extracted and superfluous features are eliminated. These deficiencies culminate in unsatisfactory accuracy and limited generalizability in real-life scenarios. In this paper, we try to tackle these challenges through three designs: (1) We present a novel framework to capture broader forgery clues by extracting multiple non-overlapping local representations and fusing them into a global semantic-rich feature. (2) Based on the information bottleneck theory, we derive Local Information Loss to guarantee the orthogonality of local representations while preserving comprehensive task-relevant information. (3) Further, to fuse the local representations and remove task-irrelevant information, we arrive at a Global Information Loss through the theoretical analysis of mutual information. Empirically, our method achieves state-of-the-art performance on five benchmark datasets. Our code is available at https://github.com/QingyuLiu/Exposing-the-Deception, hoping to inspire researchers.

Over the past years, Machine Learning-as-a-Service (MLaaS) has received a surging demand for supporting Machine Learning-driven services to offer revolutionized user experience across diverse application areas. MLaaS provides inference service with low inference latency based on an ML model trained using a dataset collected from numerous individual data owners. Recently, for the sake of data owners' privacy and to comply with the "right to be forgotten (RTBF)" as enacted by data protection legislation, many machine unlearning methods have been proposed to remove data owners' data from trained models upon their unlearning requests. However, despite their promising efficiency, almost all existing machine unlearning methods handle unlearning requests independently from inference requests, which unfortunately introduces a new security issue of inference service obsolescence and a privacy vulnerability of undesirable exposure for machine unlearning in MLaaS.

Recent years have witnessed a dramatic growing trend of Virtual YouTubers (VTubers) as a new business on social media, such as YouTube, Twitch, and TikTok. However, a significant challenge arises when VTuber voice actors face health issues or retire, jeopardizing the continuity of their avatar’s recognizable voices. A potential solution reminiscent of Conan’s Bow Tie voice changer in the popular animation Case Closed (i.e., Detective Conan) has inspired our work. To make this a reality, we introduce VTuberBowTie, a user-friendly streaming voice conversion system for real-time VTuber livestreaming. We propose an innovative streaming voice conversion approach that tackles the challenges of limited context modeling and bidirectional context dependence inherent to conventional real-time voice conversion. Rather than individually processing the voice stream in data chunks, our approach adopts a fully sequential structure that leverages contextual information preceding the input chunk, thereby expanding the perceptual range and enabling seamless concatenation. Moreover, we developed a ready-to-use interaction interface for VTuberBowTie and deployed it on various computing platforms. The experimental results show that VTuberBowTie can achieve high-quality voice conversion in a streaming manner with a latency of 179.1ms on CPU and 70.8ms on GPU while providing users a friendly interactive experience.

LiDAR sensors measure the environment by emitting lasers and, when combined with deep neural networks (DNNs), can effectively identify surrounding obstacles such as vehicles and pedestrians. Given its crucial role in autonomous driving perception, the security of LiDAR is closely tied to driving safety. Some studies have explored its vulnerabilities to physical-world attacks, such as laser-based attacks or adversarial objects. However, these methods are either extremely difficult to execute or lack stealth and flexibility. In this paper, we propose a novel attack method called EMTrig, which leverages common roadside objects combined with controlled intentional electromagnetic interference (IEMI) targeting specific LiDARs to create flexible and covert adversarial attacks against designated vehicles. This causes the victim vehicle to misidentify roadside objects as obstacles, such as other vehicles, leading to dangerous driving behaviors like sudden stops and lane changes. Unlike conventional adversarial examples, our deployed objects are common items (e.g., signboards) that are harmless without the IEMI trigger but pose a threat only under IEMI attacks, providing better stealthiness and flexibility. Extensive experiments in both digital and physical domains validate the effectiveness of EMTrig, demonstrating its significant threat to LiDAR perception.

Extensive research has revealed that adversarial examples (AE) pose a significant threat to voice-controllable smart devices. Recent studies have proposed black-box adversarial attacks that require only the final transcription from an automatic speech recognition (ASR) system. However, these attacks typically involve many queries to the ASR, resulting in substantial costs. Moreover, AE-based adversarial audio samples are susceptible to ASR updates. In this paper, we identify the root cause of these limitations, namely the inability to construct AE attack samples directly around the decision boundary of deep learning (DL) models. Building on this observation, we propose ALIF, the first black-box adversarial linguistic feature-based attack pipeline. We leverage the reciprocal process of text-to-speech (TTS) and ASR models to generate perturbations in the linguistic embedding space where the decision boundary resides. Based on the ALIF pipeline, we present the ALIF-OTL and ALIF-OTA schemes for launching attacks in both the digital domain and the physical playback environment on four commercial ASRs and voice assistants. Extensive evaluations demonstrate that ALIF-OTL and -OTA significantly improve query efficiency by 97.7% and 73.3%, respectively, while achieving competitive performance compared to existing methods. Notably, ALIF-OTL can generate an attack sample with only one query. Furthermore, our test-of-time experiment validates the robustness of our approach against ASR updates.

Ubiquitous microphones on smart devices considerably raise users’ concerns about speech privacy. Since the microphones are primarily controlled by hardware/software developers, profit-driven organizations can easily collect and analyze individuals’ daily conversations on a large scale with deep learning models, and users have no means to stop such privacy-violating behavior. In this article, we propose UniAP to empower users with the capability of protecting their speech privacy from the large-scale analysis without affecting their routine voice activities. Based on our observation of the recognition model, we utilize adversarial learning to generate quasi-imperceptible perturbations to disturb speech signals captured by nearby microphones, thus obfuscating the recognition results of recordings into meaningless contents. As validated in experiments, our perturbations can protect user privacy regardless of what users speak and when they speak. The jamming performance stability is further improved by training optimization. Additionally, the perturbations are robust against noise removal techniques. Extensive evaluations show that our perturbations achieve successful jamming rates of more than 87% in the digital domain and at least 90% and 70% for common and challenging settings, respectively, in the real-life chatting scenario. Moreover, our perturbations, solely trained on DeepSpeech, exhibit good transferability over other models based on similar architecture.

Online voice communications are widely used nowadays. To protect speech from leakage, people tend to initiate the talk in sound-isolated environments. In this paper, we reveal a novel attack that recovers high-quality speech from outside soundproof zones. The rationale of the attack is to leverage sound-sensitive characteristics of piezoelectric materials, i.e., a piezo film that can change the phase of reflected mmWaves when placed in a sound field. If the attacker transmits mmWaves and analyzes reflected signals from the piezo film, the speech information can be compromised. More importantly, the piezo film is paper-like and works without a power supply. We propose a new speech recovery methodology to transform sound waves into wireless signals and build an end-to-end eavesdropping system working as a through-wall “microphone” to recover high-quality speech stealthily. To combat signal attenuation and improve speech quality, we develop a speech-enhancement scheme based on generative adversarial networks and propose to use multi-antenna information for intelligible speech reconstruction. We conduct extensive experiments to evaluate the system. The results indicate that the system achieves over 98% accuracy for digit recognition and works well over 5m away through the wall. We also test the system under complex scenarios and give countermeasures.

Device fingerprints can satisfy the high-security requirement of modern mobile applications (e.g., mobile payments) by guaranteeing the operation is performed on a trusted device. However, existing works on device fingerprints are weak to leakage, which leads to an irreversible failure of the device fingerprint authentication system after suffering from fingerprint theft attacks. The vulnerability drives us to propose a reconfigurable device fingerprint, i.e.,MotoPrint, that can recover the system after suffering from such attacks.MotoPrintstems from the motor vibration that can represent in both signals of the accelerometer and the gyroscope (i.e., they are homologous motion signals). Therefore, we designed a two-path feature extracting network and a sensor-independent training strategy to eliminate sensor noise that can decline authentication performance. In addition,MotoPrinthas a complete reconfiguration mechanism to cope with fingerprint leakage, which brings the damaged authentication system back to health. The evaluation of 80 stand-alone vibration motors and 20 in-built ones shows thatMotoPrintcan achieve high authentication accuracy of 98.5%. Meanwhile, we also demonstrate the reconfiguredMotoPrint, which can also effectively indicate the device's uniqueness with over 98% accuracy, is independent ofMotoPrints under other stimulating codes.

Inaudible command injection transmits inaudible ultrasounds to inject adversarial speech commands into a voice assistant, therefore manipulating voice control systems (e.g., a garage door or a security camera) for illegitimate purposes. Although the attack is inaudible, we find it does leave visible “footprints”. Such attack “footprints” are the side product due to the interaction between the attack signal (i.e., input) and the acoustic components (i.e., transfer function), so they reflect the hardware characteristics of the sound capture system, including the microphone diaphragm, the low-pass filter, and the analog-to-digital converter. Moreover, unlike the non-linearity distortion that is erasable with signal-shaping techniques, the “footprints” are indelible because they are unrelated to the content of injected commands. We discover two types of indelible “footprints” embedded in the recording spectrogram, namely abnormal interfering noise and abnormal demodulation. A software-based detection method and a portable detector, DolphinTag, are further designed to identify these “footprints”. The software-based method achieves a detection accuracy of 99.8% on the phone models exhibiting abnormal interfering noise, and our DolphinTag achieves 100% detection accuracy which detects the ultrasound attack by actively facilitating the abnormal demodulation.

As one of the most representative applications built on deep learning, audio systems, including keyword spotting, automatic speech recognition, and speaker identification, have recently been demonstrated to be vulnerable to adversarial examples, which have already raised general concerns in both academia and industry. Existing attacks follow the same adversarial example generation paradigm from computer vision, i.e., overlaying the optimized additive perturbations on original voices. However, due to the additive perturbations’ nature on human audibility, balancing the stealthiness and attack capability remains a challenging problem. In this paper, we rethink the stealthiness of audio adversarial examples and turn to introduce another kind of audio distortion, i.e., reverberation, as a new perturbation format for stealthy adversarial example generation. Such convolutional adversarial perturbations are crafted as real-world impulse responses and behave as a natural reverberation for deceiving humans. Based on this idea, we propose AdvReverb to construct, optimize, and deliver phoneme-level convolutional adversarial perturbations on both speech and music carriers with a well-designed objective. Experimental results demonstrate that AdvReverb could realize high attack success rates over 95% on three audio-domain tasks while achieving superior perceptual quality and keeping stealthy from human perception in over-the-air and over-the-line delivery scenarios.

Despite their prevalence and indispensability in the perception modules of autonomous vehicles, cameras have shown susceptibility to numerous attacks. Among them, the phantom spoofing attack is of significant concern. In such attacks, malefactors employ electronic display devices like projectors and display monitors to generate deceptive objects, thereby duping the object detectors in autonomous vehicles. However, existing detection methodologies are narrowly focused on a single device category, ignoring the multitude of devices that could be leveraged for attacks. Furthermore, the artificial modality-based solution presently in use lacks efficacious fusion mechanisms. In response to these limitations, we propose PhaDe, a practical deep learning-based system adept at detecting phantom spoofing attacks from a variety of and even unfamiliar attack devices. Our approach introduces two image processing techniques to construct artificial modalities and further advances a multi-head self-attention MSA-based fusion module for more versatile integration of disparate modalities. To boost the generalization capacity of our system against novel, unseen attacks, we incorporate two representation-level losses to align feature distributions from various domains. Evaluations conducted on our own dataset, encompassing fake objects from several device types, attest to the efficacy of our system. Our results indicate an accuracy of 98.80% on familiar domains and a detection success rate of 94.03% on unfamiliar domains. Additionally, PhaDe demonstrates a swift response time, fulfilling the practicality requisites.

Federated Learning (FL) is a promising distributed learning approach that enables multiple clients to collaboratively train a shared global model. However, recent studies show that FL is vulnerable to various poisoning attacks, which can degrade the performance of global models or introduce backdoors into them. In this paper, we first conduct a comprehensive study on prior FL attacks and detection methods. The results show that all existing detection methods are only effective against limited and specific attacks. Most detection methods suffer from high false positives, which lead to significant performance degradation, especially in not independent and identically distributed (non-IID) settings. To address these issues, we propose FLTracer, the first FL attack provenance framework to accurately detect various attacks and trace the attack time, objective, type, and poisoned location of updates. Different from existing methodologies that rely solely on cross-client anomaly detection, we propose a Kalman filter-based cross-round detection to identify adversaries by seeking the behavior changes before and after the attack. Thus, this makes it resilient to data heterogeneity and is effective even in non-IID settings. To further improve the accuracy of our detection method, we employ four novel features and capture their anomalies with the joint decisions. Extensive evaluations show that FLTracer achieves an average true positive rate of over 96.88% at an average false positive rate of less than 2.67%, significantly outperforming SOTA detection methods (https://github.com/Eyr3/FLTracer).

Covert eavesdropping via microphones has always been a major threat to user privacy. Benefiting from the acoustic non-linearity property, the ultrasonic microphone jammer (UMJ) is effective in resisting this long-standing attack. However, prior UMJ researches underestimate adversary's attacking capability in reality and miss critical metrics for a thorough evaluation. The strong assumptions of adversary unable to retrieve information under low word recognition rate, and adversary's weak denoising abilities in the threat model make these works overlook the vulnerability of existing UMJs. As a result, their UMJs' resilience is overestimated. In this paper, we refine the adversary model and completely investigate potential eavesdropping threats. Correspondingly, we define a total of 12 metrics that are necessary for evaluating UMJs' resilience. Using these metrics, we propose a comprehensive framework to quantify UMJs' practical resilience. It fully covers three perspectives that prior works ignored to some degree, i.e., ambient information, semantic comprehension, and collaborative recognition. Guided by this framework, we can thoroughly and quantitatively evaluate the resilience of existing UMJs towards eavesdroppers. Our extensive assessment results reveal that most existing UMJs are vulnerable to sophisticated adverse approaches. We further outline the key factors influencing jammers' performance and present constructive suggestions for UMJs' future designs.

In cyber-physical systems, inertial sensors are the basis for identifying motion states and making actuation decisions. However, extensive studies have proved the vulnerability of those sensors under acoustic transduction attacks, which leverage malicious acoustics to trigger sensor measurement errors. Unfortunately, the threat from such attacks is not assessed properly because of the incomplete investigation on the attack's potential, especially towards multiple-degree-of-freedom systems, e.g., drones. To thoroughly explore the threat of acoustic transduction attacks, we revisit the attack model and design a new yet practical acoustic modulation-based attack, named KITE. Such an attack enables stable and controllable injections, even under frequency offset based distortions that limit the effect of prior attacking approaches. KITE exploits the potential threat of transduction attacks without the need of strengthening attackers' abilities. Furthermore, we extend the attack surface to multiple-degree-of-freedom (MDOF) systems, which are more widely deployed but ignored by prior work. Our study also covers the scenario of attacking moving targets. By revealing the practical threat from acoustic transduction attacks, we appeal for both the attention to their harm and necessary countermeasures.

Current facial authentication (FA) systems are mostly based on the images of human faces, thus suffering from privacy leakage and spoofing attacks. Mainstream systems utilize facial geometry features for spoofing mitigation, but they are still vulnerable to feature manipulation, e.g., 3D-printed human faces. In this article, we propose a novel privacy-preserving anti-spoofing FA system, named RFace, which extracts both the 3D geometry and inner biomaterial features of faces using a COTS RFID tag array. These features are difficult to obtain and forge, hence are resistant to spoofing attacks. Unlike images, RF signals are not perceptible to human eyes, so RFace protects user's privacy. We build a theoretical model to rigorously prove the feasibility of feature acquisition and the correlation between facial features and RF signals. To enhance the security of RFace, we specify the tag reading order for each authentication to defend against the signal replay attack. For practicality, we design an effective algorithm to mitigate the impact of unstable distance and angle deflection from the face to the array. Extensive experiments with 30 participants and three types of spoofing attacks show that RFace achieves an average authentication success rate of over 95.7$\%$and an EER of 4.4$\%$. More importantly, no replay attack or spoofing attack succeeds in deceiving RFace in the experiments.

Among the foremost contributors to compromised driving safety is the abrupt emergence of obstacles or pedestrians within drivers’ non-line-of-sight regions. Previous investigations into non-line-of-sight imaging have predominantly depended on costly apparatus or have been confined to controlled laboratory settings (e.g., extensive planar reflectors and regulated illumination). Consequently, these technological approaches prove impractical within intricate driving environments. In this paper, we introduce a shadow based non-line-of-sight moving obstacle detection system devised to augment Advanced Driver Assistance Systems (ADAS), ensuring adequate time for safe response and halting. Our approach incorporates a shadow signal discriminator tailored to evaluate faint shadows generated by moving obstacles, such as pedestrians within blind spots. Note that we merely use commercial onboard sensors and our system is robust to various lighting scenarios and planar reflectors. We comprehensively assess our methodology's adaptability by employing datasets acquired from real-world driving scenarios encompassing diverse road surfaces and lighting conditions. The results substantiate the system's efficacy in detecting pedestrians in motion within NLOS regions, showcasing an impressive detection range of 22 meters. This proficiency enables the system to pre-emptively forewarn the ADAS, facilitating the maintenance of a safe distance from the pedestrian.

Recent years have witnessed enormous research efforts on WiFi sensing to enable intelligent services of Internet of Things. However, due to the omni-directional broadcasting manner of WiFi signals, the activity semantic underlying the signals can be leaked to adversaries for surveillance, as demonstrated by our previous work. In this paper, we further extend the attack capability ofActListenerto impersonation attack, which could eavesdrop on users’ behavioral uniqueness imperceptibly using a WiFi infrastructure in any location of user sensing area. In particular,ActListener detects each human activityand converts the eavesdropped signals to that by legitimate devices based on our proposed signal propagation models. To extract noise-resilient individual behavioral uniqueness from converted CSI of WiFi signals, we further add user identification models into the substitute model set for training the signal pattern calibration generative model. Experimental results demonstrate thatActListenercould achieve over 80% accuracy in activity semantics retrieval and impersonation by using the converted signals.

As automatic speech recognition evolves, deployment of the voice user interface (VUI) has boomingly expanded. Especially since the COVID-19 pandemic, the VUI has gained more attention in online communication owing to its non-contact property. However, the VUI struggles to be applied in public scenes due to the degradation of received audio signals caused by various ambient noises. In this article, we propose Wavoice , the first noise-resistant multi-modal speech recognition system that fuses two distinct voices sensing modalities (i.e., millimeter-wave signals and audio signals from a microphone) together. One key contribution is to model the inherent correlation between millimeter-wave and audio signals. Based on it, Wavoice facilitates the real-time noise-resistant voice activity detection and user targeting from multiple speakers. Additionally, we elaborate on two novel modules for multi-modal fusion embedded into the neural network, leading to accurate speech recognition. Extensive experiments prove the effectiveness of Wavoice under adverse conditions—that is, the character recognition error rate below 1% in a range of 7 m. In terms of robustness and accuracy, Wavoice considerably outperforms existing audio-only speech recognition methods with lower character error and word error rates.

Physically unclonable hardware fingerprints can be used for device authentication. The photo-response non-uniformity (PRNU) is the most reliable hardware fingerprint of digital cameras and can be conveniently extracted from images. However, we find image post-processing software may introduce extra noise into images. Part of this noise remains in the extracted PRNU fingerprints and is hard to be eliminated by traditional approaches, such as denoising filters. We define this noise as software noise, which pollutes PRNU fingerprints and interferes with authenticating a camera armed device. In this paper, we propose novel approaches for fingerprint matching, a critical step in device authentication, in the presence of software noise. We calculate the cross correlation between PRNU fingerprints of different cameras using a test statistic such as the Peak to Correlation Energy (PCE) so as to estimate software noise correlation. During fingerprint matching, we derive the ratio of the test statistic on two PRNU fingerprints of interest over the estimated software noise correlation. We denote this ratio as the fingerprint to software noise ratio (FITS), which allows us to detect the PRNU hardware noise correlation component in the test statistic for fingerprint matching. Extensive experiments over 10,000 images taken by more than 90 smartphones are conducted to validate our approaches, which outperform the state-of-the-art approaches significantly for polluted fingerprints. We are the first to study fingerprint matching with the existence of software noise.

This paper proposes a novel Deconvolution-enhanced data Augmentation method, DeAug, for ultrasonic-based speaker verification anti-spoofing systems to detect the liveness of voice sources in physical access, which aims to improve the performance of liveness detection on unseen devices where no data is collected yet. Specifically, DeAug first employs a wiener deconvolution pre-processing on available collected data to generate enhanced clean signal samples. Then, the generated samples are convolved with different device impulse responses, to enable the signal with the unseen devices' channel characteristics. Experiments on cross-domain datasets show that our proposed augmentation method can improve the performance of ultrasonic-based anti-spoofing systems by 97.8% relatively, and a further improvement of up to 43.4% can be obtained after applying domain adversarial training on multi-device augmented data.

Face recognition has made significant advances with the development of deep learning and has begun to be deployed in some unrestricted scenarios. Many smartphones, for example, have infrared sensors that allow them to capture clear images even in low-light conditions. Face authentication under complex environmental conditions can thus be accomplished by matching NIR-VIS face images across modalities. However, existing NIR-VIS datasets lack enough variation in face attributes and are insufficient for real-world scenarios. To address the aforementioned issues, we first propose a 300-person NIR-VIS cross-modality face dataset with a variety of attributes. Based on modal information removal, we proposed a NIR-VIS cross-modal face recognition model. We can effectively extract modal information by constraining the similarity distribution of modalities and then using the orthogonal loss to remove modal information from identity features. The method achieves excellent results on our dataset and CASIA NIR-VIS 2.0 dataset.

The malicious use and widespread dissemination of deepfake pose a significant crisis of trust. Current deepfake detection models can generally recognize forgery images by training on a large dataset. However, the accuracy of detection models degrades significantly on images generated by new deepfake methods due to the difference in data distribution. To tackle this issue, we present a novel incremental learning framework that improves the generalization of deepfake detection models by continual learning from a small number of new samples. To cope with different data distributions, we propose to learn a domain-invariant representation based on supervised contrastive learning, preventing overfit to the insufficient new data. To mitigate catastrophic forgetting, we regularize our model in both feature-level and label-level based on a multi-perspective knowledge distillation approach. Finally, we propose to select both central and hard representative samples to update the replay set, which is beneficial for both domain-invariant representation learning and rehearsal-based knowledge preserving. We conduct extensive experiments on four benchmark datasets, obtaining the new state-of-the-art average forgetting rate of 7.01 and average accuracy of 85.49 on FF++, DFDC-P, DFD, and CDF2. Our code is released at \textcolorblue https://github.com/DeepFakeIL/DFIL.

Deepfake has taken the world by storm, triggering a trust crisis. Current deepfake detection methods are typically inadequate in generalizability, with a tendency to overfit to image contents such as the background, which are frequently occurring but relatively unimportant in the training dataset. Furthermore, current methods heavily rely on a few dominant forgery regions and may ignore other equally important regions, leading to inadequate uncovering of forgery cues.

As the demand for precision medicine rapidly grows, companion diagnostics is proposed to monitor and evaluate therapeutic effects for adjusting medicine plans in time. Although a set of clinical companion diagnostics tools (e.g., polymerase chain reaction) have been investigated, they are expensive and only accessible in a lab environment, which hinders the promotion to broader patients. In light of this situation, we take the first steps towards developing a real-world companion diagnostic tool by leveraging mobile technology. In this paper, we present TherapyPal, a privacy-preserving medicine effectiveness computational framework by harnessing semantic hashing-based digital symptomatic phenotyping. Specifically, sensor data captured from daily-life activities is first transformed into spectrograms. Then, we develop a hashing learning network to extract privacy-masked symptomatic phenotypes on smartphones. Afterward, symptomatic hashes at different medicine states are fed to a contrastive learning network in the cloud for treatment effectiveness detection. To evaluate the performance, we conduct a clinical study among 65 Parkinson's disease (PD) patients under dopaminergic drug treatment. The results show that TherapyPal can achieve around 84.1% medicine effectiveness detection accuracy among patients and above 0.925 privacy-masked scores for protecting each private attribute, which validates the reliability and security of TherapyPal to be used as a real-world companion diagnostics tool.

With the broad integration of deep learning in Speaker Recognition (SR) systems, adversarial example attacks have been a significant threat raising user security concerns. Nevertheless, recent studies demonstrate that using input transformations (e.g., re-quantization, resampling, bandpass filtering) as a low-cost prefilter can efficiently mitigate such adversarial example attacks. These prefilters constrain the injection space of adversarial perturbations in both time and frequency domains, leading to either degraded attack performance or amplified perturbation noise. This paper proposes a new adversarial example attack, BypTalker, which could bypass these prefilter-enabled SR systems while remaining imperceptible to human listeners. BypTalker employs ensemble learning with diverse substitute pre-filters in the training phase to enhance the adversarial example’s adaptiveness to different prefilters. Furthermore, it incorporates an Acoustic Masker to cloak adversarial perturbations based on psychoacoustics effectively. This masker is well selected from a proposed metric M-Sup for minimizing the perturbation’s auditory to human perception. Experimental results show that BypTalker can achieve an Attack Success Rate of 99.1% and a Perceptual Evaluation of Speech Quality of 4.32, respectively.

Fingerprint recognition has been a vital security guard for various applications whose vulnerability has been explored by different works. However, previous works on spoofing fingerprint recognition rely on prior knowledge (e.g., photos and minutiae) of the target fingerprint, which fails to implement in practical scenarios. In this paper, we design a fingerprint spoofing attack, namely FingerFaker, to explore the vulnerability of fingerprint recognition, which can spoof automated fingerprint recognition systems (AFRSs) without prior knowledge of target fingerprints. Specifically, we propose a novel concept of "pseudo-minutiae-set" as an effective optimization object and design a two-stage scheme to optimize "pseudo-minutiaeset" leveraging a two-factor evolutionary strategy. In addition, we use a GAN-based training strategy with a minutiae loss function to pre-train a fingerprint generator to map a "pseudo-minutiae-set" into a fingerprint. We use 6342 fingerprint images to verify the performance of FingerFaker on spoofing the open-source AFRS, which shows a high attack success rate (ASR) of 97.78%. Meanwhile, we conduct a realistic case study on commercial off-the-shelf (COTS) AFRS, where FingerFaker also shows 94.22% ASR. Finally, we explore the impact of different conditions to guide the attack and propose countermeasures to mitigate the harm.

One of the most serious factors compromising driving safety is when people in drivers' non-line-of-sight areas rush out suddenly. Existing studies on non-line-of-sight imaging rely on expensive equipment or are limited to severe laboratory conditions (e.g., massive planar reflectors and controlled illumination), rendering these technologies inapplicable in complex driving scenarios. In this paper, we propose a non-line-of-sight moving obstacle detection system Ghost-Probe, which can provide an advanced driver assistance system (ADAS) with sufficient time to respond and stop safely. We design a shadow signal discriminator to assess the weak shadows created by a moving obstacle, such as pedestrians in the blind area, while simultaneously filtering out the impacts of other complicated illumination. Note that we merely use commercial monocular cameras and our system is robust to a wide range of lighting scenarios and planar reflectors. We evaluate the generalizability of our approach using the datasets collected in real-world driving scenarios with a variety of road surface and lighting circumstances. The results indicate that our system can detect the moving pedestrian in the non-line-of-sight area at a distance of 20 meters and offer the ADAS system advance warning to keep a safe distance.

An audio system containing loudspeakers and microphones is the fundamental hardware for voice-enabled devices, enabling voice interaction with mobile applications and smart homes. This paper presents MagBackdoor, the first magnetic field attack that injects malicious commands via a loudspeaker-based backdoor of the audio system, compromising the linked voice interaction system. MagBackdoor focuses on the magnetic threat on loudspeakers and manipulates their sound production stealthily. Consequently, the microphone will inevitably pick up malicious sound generated by the attacked speaker, due to the closely packed arrangement of internal audio systems. To prove the feasibility of MagBackdoor, we conduct comprehensive simulations and experiments. This study further models the mechanism by which an external magnetic field excites the sound production of loudspeakers, giving theoretical guidance to MagBackdoor. Aiming at stealthy magnetic attacks in real-world scenarios, we self-design a prototype that can emit magnetic fields modulated by voice commands. We implement MagBackdoor and evaluate it across a wide range of smart devices involving 16 smartphones, four laptops, two tablets, and three smart speakers, achieving an average 95% injection success rate with high-quality injected acoustic signals.

With the increasing deployment of voice-controlled devices in homes and enterprises, there is an urgent demand for voice identification to prevent unauthorized access to sensitive information and property loss. However, due to the broadcast nature of sound wave, a voice-only system is vulnerable to adverse conditions and malicious attacks. We observe that the cooperation of millimeter waves (mmWave) and voice signals can significantly improve the effectiveness and security of user identification. Based on the properties, we propose a multi-modal user identification system (named WavoID) by fusing the uniqueness of mmWave-sensed vocal vibration and mic-recorded voice of users. To estimate fine-grained waveforms, WavoID splits signals and adaptively combines useful decomposed signals according to correlative contents in both mmWave and voice. An elaborated anti-spoofing module in WavoID comprising biometric bimodal information defend against attacks. WavoID produces and fuses the response maps of mmWave and voice to improve the representation power of fused features, benefiting accurate identification, even facing adverse circumstances. We evaluate WavoID using commercial sensors on extensive experiments. WavoID has significant performance on user identification with over 98% accuracy on 100 user datasets.

The proliferation of deepfake content has motivated a surge of detection studies. However, existing detection methods in the audio area exclusively work in English, and there is a lack of data resources in other languages. Cross-lingual deepfake detection, a critical but rarely explored area, urges more study. This paper conducts the first comprehensive study on the cross-lingual perspective of deepfake detection. We observe that English data enriched in deepfake algorithms can teach a detector the knowledge of various spoofing artifacts, contributing to performing detection across language domains. Based on the observation, we first construct a first-of-its-kind cross-lingual evaluation dataset including heterogeneous spoofed speech uttered in the two most widely spoken languages, then explored domain adaptation (DA) techniques to transfer the artifacts detection capability and propose effective and practical DA strategies fitting the cross-lingual scenario. Our adversarial-based DA paradigm teaches the model to learn real/fake knowledge while losing language dependency. Extensive experiments over 137-hour audio clips validate the adapted models can detect fake audio generated by unseen algorithms in the new domain.

With the continuing growth of voice-controlled devices, voice metrics have been widely used for user identification. However, voice biometrics is vulnerable to replay attacks and ambient noise. We identify that the fundamental vulnerability in voice biometrics is rooted in its indirect sensing modality (e.g., microphone). In this paper, we presentVocalPrint, a resilient mmWave interrogation system which directly captures and analyzes the vocal vibrations for user authentication. Specifically,VocalPrintexploits the unique disturbance of the skin-reflect radio frequency (RF) signals around the near-throat region of the user, caused by the vocal vibrations. The complex ambient noise is isolated from the RF signal using a novel resilience-aware clutter suppression approach for preserving fine-grained vocal biometric properties. Afterward, we extract the vocal tract and vocal source features and input them into an ensemble classifier for authentication.VocalPrintis practical as it allows the effortless transition to a smartphone while having sufficient usability due to its non-contact nature. Our experimental results from 41 participants with different interrogation distances, orientations, and body motions show thatVocalPrintachieves over 96 percent authentication accuracy even under unfavorable conditions. We demonstrate the resilience of our system against complex noise interference and spoof attacks of various threat levels.

Incompatible protocols and electromagnetic interference obstruct the realization of an everything-connected Internet of Things (IoT) communication network. Our system, Deaf-Aid, utilizes a stealthy speaker-to-gyroscope channel to build robust communication. Compared with existing solutions adopting physical covert channels, Deaf-Aid is free from the limitations of manual receiver distinction, additional hardware, conditional placement, or physical contact. It exploits ultrasounds to force gyroscopes embedded in receivers to resonate, so as to convey information. We investigate the relationship among axes in a gyroscope to deal with frequency offset and support multi-channel communication. Meanwhile, receivers are identified automatically via device fingerprints consisting of diversity of gyroscopes’ resonant frequency ranges. Furthermore, we enable Deaf-Aid the capability of mobile communication, which is an essential demand for IoT devices. We address the challenge of recovering accurate signals from motion interference. Extensive evaluations, including that on the commercial off-the-shelf devices, demonstrate that Deaf-Aid yields 47 bps with BER below 1%. To our best knowledge, Deaf-Aid is the first work to enable stealthy mobile IoT communication based on inertial sensors.

Multi-factor user authentication becomes increasingly popular due to its superior security comparing with single-factor user authentication. However, existing multi-factor user authentication methods usually require multiple interactions between users and different authentication components when inputting the multiple factors, leading to extra overhead and bad user experience. In this paper, we propose a secure and user-friendly multi-factor user authentication system named BioDraw. It utilizes four categories of biometrics (impedance, geometry, behavior, and composition) of human hand plus the pattern-based password to identify and authenticate users. User only needs to draw a pattern on a radio frequency identification tag array, while four biometrics can be collected simultaneously. Specifically, we first design a gradient-based pattern recognition algorithm to precisely extract user’s secret pattern. Then, a convolutional neural network- and long short-term memory-based classifier is utilized for user recognition. Furthermore, to guarantee the systemic security, an anti-replay method called Binary ALOHA is proposed to detect replayed signals. We conduct extensive experiments with 30 volunteers. The experiment results show that BioDraw can achieve high authentication accuracy (with a 2%– false reject rate) and is effective in defending against various attacks.

Recent years have witnessed enormous research efforts on WiFi sensing to enable intelligent services of Internet of Things. However, due to the omni-directional broadcasting manner of WiFi signals, the activity semantic underlying the signals is leaked to adversaries for surveillance in all probability. To reveal the threat, this paper demonstrates ActListener, which could eavesdrop on user activities imperceptibly using a WiFi infrastructure in any location of user sensing area. The proposed attack requires no direct physical access to the victim user’s devices and prior knowledge of activity recognition model details and device locations. In particular, ActListener first detects the signal segment induced by each human activity, and estimates the locations of legitimate devices and the victim users relative to the adversary’s device for further signal modeling. Then, ActListener models propagating WiFi signals to construct the relationship between physical locations and received signals, and converts the eavesdropped signals to that by legitimate devices based on the models. Furthermore, a neural network-based generative model is designed to calibrate the converted signals for resisting noises in over-the-air WiFi signals. Experiments show ActListener achieves 88.4% average α-similarity on recovering originally signals from eavesdropped ones, and over 90% accuracy in activity recognition.

Fingerprint recognition technology is the most widely used technology in the field of biometrics and is also the preferred solution for mobile devices and the financial industry. However, traditional fingerprint recognition technology requires expensive sensors on the one hand and risks leaking fingerprint images on the other hand. In this paper, we propose Fingersound, a low-cost and deployable authentication system that utilizes the sound generated by finger sliding as an identity feature. We use a microphone array to record continuous sliding sound and extract multiple features in the frequency and time domains. Then we use various algorithms including a deep neural network to perform user authentication. We also design a mobile phone application that can interact with the embedded system to manage users and record authentication history. In our experiments, 100 participants used this system and achieved an equal error rate of 5.2%. Additionally, we investigate the system’s robustness within different sliding materials, texturesand under noise disturbances. We further demonstrate the resistance of Fingersound to replay attack.

Covert eavesdropping via microphones has always been a major threat to user privacy. Benefiting from the acoustic non-linearity property, the ultrasonic microphone jammer (UMJ) is effective in resisting this long-standing attack. However, prior UMJ researches underestimate adversary’s attacking capability in reality and miss critical metrics for a thorough evaluation. The strong assumptions of adversary unable to retrieve information under low word recognition rate, and adversary’s weak denoising abilities in the threat model make these works overlook the vulnerability of existing UMJs. As a result, their UMJs’ resilience is overestimated. In this paper, we refine the adversary model and completely investigate potential eavesdropping threats. Correspondingly, we define a total of 12 metrics that are necessary for evaluating UMJs’ resilience. Using these metrics, we propose a comprehensive framework to quantify UMJs’ practical resilience. It fully covers three perspectives that prior works ignored in some degree, i.e., ambient information, semantic comprehension, and collaborative recognition. Guided by this framework, we can thoroughly and quantitatively evaluate the resilience of existing UMJs towards eavesdroppers. Our extensive assessment results reveal that most existing UMJs are vulnerable to sophisticated adverse approaches. We further outline the key factors influencing jammers’ performance and present constructive suggestions for UMJs’ future designs.

More and more people turn to online voice communication with loudspeaker-equipped devices due to its convenience. To prevent speech leakage, soundproof rooms are often adopted. This paper presents mmPhone, a novel acoustic eavesdropping system that recovers loudspeaker speech protected by soundproof environments. The key idea is that properties of piezoelectric films in mmWave band can change with sound pressure due to the piezoelectric effect. If the property changes are acquired by an adversary (i.e., characterizing the piezoelectric effect with mmWaves), speech leakage can happen. More importantly, the piezoelectric film can work without a power supply. Base on this, we proposed a methodology using mmWaves to sense the film and decoding the speech from mmWaves, which turns the film into a passive "microphone". To recover intelligible speech, we further develop an enhancement scheme based on a denoising neural network, multi-channel augmentation, and speech synthesis, to compensate for the propagation and penetration loss of mmWaves. We perform extensive experiments to evaluate mmPhone and conduct digit recognition with over 93% accuracy. The results indicate mmPhone can recover high-quality and intelligible speech from a distance over 5m and is resilient to incident angles of sound waves (within 55 degrees) and different types of loudspeakers.

Faced with the threat of identity leakage during voice data publishing, users are engaged in a privacy-utility dilemma while enjoying convenient voice services. Existing studies employ direct modification or text-based re-synthesis to de-identify users' voices, but resulting in inconsistent audibility for human participants and not adaptive to informed attacks. In this poster, we propose a non-intrusive and adaptive speaker de-identification scheme to balance the privacy and utility of voice services. We generate adversarial examples to conceal user identity from exposure by Automatic Speaker Identification (ASI). By learning a compact distribution with a conditional variational auto-encoder, our system enables on-demand target sampling and diverse identity transformation. We also introduce the acoustic masking effect to construct inaudible perturbations, thus preserving the speech content and perceptual quality. Experiments on 50 speakers show our system could achieve 98.2% successful de-identification on 4 mainstream ASIs with an objective perceptual quality of 4.38 and a subjective mean opinion score of 4.56.

Earpiece mode of smartphones is often used for confidential communication. In this paper, we proposed a remote(>2m) and motion-resilient attack on smartphone earpiece. We developed an end-to-end eavesdropping system mmEve based on a commercial mmWave sensor to recover speech emitted from smartphone earpiece. The rationale of the attack is based on our observation that, soundwaves emitted from the smartphone's earpiece have a strong correlation with reflected mmWaves from the smartphone's rear. However, we find the recovered speech suffers from the sensor's self-noise and smartphone user's motion which limit attack distance to less than 2m, causing limited threats in real world. We modeled the motion interference under mmWave sensing and proposed a motion-resilient solution by optimizing the fitting function on I/Q plane. To achieve a practical attack with reasonable attack distance, we developed a GAN-based denoising scheme to eliminate the noise pattern of the sensor, which boosted the attack range to 6--8m. We evaluated mmEve with extensive experiments and find 23 different models of smartphones manufactured by Samsung, Huawei, etc. can be compromised by the proposed attack.

The integration of deep learning on Speaker Recognition (SR) advances its development and wide deployment, but also introduces the emerging threat of adversarial examples. However, only a few existing studies investigate its practical threat in physical domain, which either evaluate its feasibility only by directly replaying generated adversarial examples, or explore the partial channel interference for robustness improvement. In this paper, we propose a physical adversarial example attack, PhyTalker, which could generate and inject perturbations on voices in a live-streaming manner on attacking various SR models in different physical channels. Compared with the typical adversarial example for digital attacks, PhyTalker generates a subphoneme-level perturbation dictionary to decouple the perturbation optimization and injection. Moreover, we introduce the channel augmentation to compensate both device and environmental distortions, as well as model ensemble to improve the perturbation transferability. Finally, PhyTalker recognizes and localizes the latest recorded phoneme to determine the corresponding perturbations for real-time broadcasting. Extensive experiments are conducted with a large-scale corpus in real physical scenarios, and results show that PhyTalker achieves an overall Attack Success Rate (ASR) of 85.5% in attacking mainstream SR systems and Mel Cepstral Distortion (MCD) of 2.45dB in human audibility.

In cyber-physical systems, inertial sensors are the basis for identifying motion states and making actuation decisions. However, extensive studies have proved the vulnerability of those sensors under acoustic transduction attacks, which leverage malicious acoustics to trigger sensor measurement errors. Unfortunately, the threat from such attacks is not assessed properly because of the incomplete investigation on the attack's potential, especially towards multiple-degree-of-freedom systems, e.g., drones. To thoroughly explore the threat of acoustic transduction attacks, we revisit the attack model and design a new yet practical acoustic modulation-based attack, named KITE. Such an attack enables stable and controllable injections, even under frequency offset based distortions that limit the effect of prior attacking approaches. KITE exploits the potential threat of transduction attacks without the need of strengthening attackers' abilities. Furthermore, we extend the attack surface to multiple-degree-of-freedom systems, which are more widely deployed but ignored by prior work. Our study also covers the scenario of attacking moving targets. By revealing the practical threat from acoustic transduction attacks, we appeal for both the attention to their harm and necessary countermeasures.

Due to the premise of uniqueness and acceptance, fingerprint has been the most adopted biometric technologies in high-impact applications (e.g., smartphone security, monetary transactions and international-border verification). Although there are an array of commercial fingerprint scanners across different sensing modalities including optical, capacitive, thermal and ultrasonic, existing fingerprint technologies are vulnerable to spoofing attacks via fake-finger in Kanget al., 2003. In this paper, we investigate a new dimension of fingerprint sensing based on the friction-excited sonic wave (in simpler words, ”voice of fingerprint”) from a user swiping his fingertip on everyday surfaces. Specifically, we developSonicPrintto leverage the intrinsic fingerprint ridge information in sonic wave for user identification. First, the complex ambient noise is isolated from the sonic wave using background isolation and adaptive segmentation models. Afterward, a series of multi-level friction descriptors that highlight the target fingerprint information is extracted. These descriptors are fed to a specially designed ensemble classifier for user identification.SonicPrintis practical as it leverages in-built microphones in smart devices, requiring no hardware modifications. As the first exploratory study, our experimental results with 31 participants over three different swipe actions on 12 different types of materials show up to a 98 percent identification accuracy.

Facial recognition has become the surge on mobile authentication scenarios and makes up a huge market share for various apps, such as MasterCard, Google Wallet, and AliPay. However, existing solutions suffer from various impersonation attacks, including photo-spoofing attack, video-replay attack, and 3D facial mask attack. State-of-the-art countermeasures either require additional user intervention or introduce specialized high-end sensors. Even introducing these extra efforts, these approaches still hardly defend the latest 3D facial mask attacks, which gradually become accessible due to the prevalence of low-cost 3D printing. In this paper, we propose an anti-spoofing facial recognition system, PassFace, which verifies the smartphone for authentication as the second factor merely using raw facial videos without any user intervention, to defeat impersonation attacks. In particular, when receiving a user’s selfie video, PassFace identifies the user’s face from the video, and meanwhile extracts the highly unique and physically irreproducible camera fingerprint, i.e., Photo Response Non-Uniformity (PRNU), built in the smartphone from key frames of the video. After that, the system compares the Peak to Correlation Energy (PCE) calculated by the estimated PRNU and the reference profile with a threshold for authentication. Experiment results demonstrate PassFace can achieve satisfactory performance in authentication and attack resistance.

Biometrics have been widely used in user authentications. However, existing outer-body biometrics (e.g., fingerprint), collecting from body surface, are vulnerable to spoofing attacks. Although inner-body biometrics, such as the electrocardiogram, are hard to be forged, their complex acquisition methods and instability lead to unsatisfactory user experience. Therefore, achieving good user-friendliness and high security simultaneously in biometric-based authentication is challenging. In this paper, we propose Hand-Key, an attack-resilient and user-friendly user authentication system to address the above challenge. Hand-Key utilizes a low-cost radio frequency identification (RFID) tag array to simultaneously collect the inner-body composition and outer-body geometric features of human hand to identify users. Users are merely required to hold their hands in a ‘handshaking’ pose between a reader's antenna and a tag array during authentication. To further enhance the security, we tactfully leverage the inherent randomness of the anti-collision scheme in RFID systems to make Hand-Key immune against replay attacks. We built a prototype of Hand-Key and conducted extensive experiments with 30 volunteers. The results show that Hand-Key achieves an authentication success rate of 99%+.

Current facial authentication (FA) systems are mostly based on the images of human faces, thus suffering from privacy leakage and spoofing attacks. Mainstream systems utilize facial geometry features for spoofing mitigation, which are still easy to deceive with the feature manipulation, e.g., 3D-printed human faces. In this paper, we propose a novel privacy-preserving anti-spoofing FA system, named RFace, which extracts both the 3D geometry and inner biomaterial features of faces using a COTS RFID tag array. These features are difficult to obtain and forge, hence are resistant to spoofing attacks. RFace only requires users to pose their faces in front of a tag array for a few seconds, without leaking their visual facial information. We build a theoretical model to rigorously prove the feasibility of feature acquisition and the correlation between the facial features and RF signals. For practicality, we design an effective algorithm to mitigate the impact of unstable distance and angle deflection from the face to the array. Extensive experiments with 30 participants and three types of spoofing attacks show that RFace achieves an average authentication success rate of over 95.7% and an EER of 4.4%. More importantly, no spoofing attack succeeds in deceiving RFace in the experiments.

With the advance in automatic speech recognition, voice user interface has gained popularity recently. Since the COVID-19 pandemic, VUI is increasingly preferred in online communication due to its non-contact. Additionally, various ambient noise impedes the public applications of voice user interfaces due to the requirement of audio-only speech recognition methods for a high signal-to-noise ratio. In this paper, we present Wavoice, the first noise-resistant multi-modal speech recognition system that fuses two distinct voice sensing modalities, i.e., millimeter-wave (mmWave) signals and audio signals from a microphone, together. One key contribution is that we model the inherent correlation between mmWave and audio signals. Based on it, Wavoice facilitates the real-time noise-resistant voice activity detection and user targeting from multiple speakers. Furthermore, we elaborate on two novel modules into the neural attention mechanism for multi-modal signals fusion, and result in accurate speech recognition. Extensive experiments verify Wavoice's effectiveness under various conditions with the character recognition error rate below 1% in a range of 7 meters. Wavoice outperforms existing audio-only speech recognition methods with lower character error rate and word error rate. The evaluation in complex scenes validates the robustness of Wavoice.

Internet-of-Things (IoT) devices are widely deployed nowadays. A large number of smart home IoT devices are hosted on a cloud server for easy management. Users can use their accounts to initiate operations and management on IoT devices through a cloud server, such as updating firmware and configuring devices. However, the cloud account may be hacked resulting in adversarial attacks to the hosted IoT devices. As a consequence, an adversary may perform malicious operations through the cloud remotely to the hosted IoT devices without user awareness. Motivated by this, in this article we propose gateway-based 2 factor authentication (G2F), a secure user authentication framework dedicated for a gateway based on the universal 2nd factor (U2F) protocol to enhance the security of IoT devices management. In G2F, the user authentication on the gateway is completed utilizing a hardware token that interacts with the local gateway node to guarantee the token owner’s presence. Furthermore, G2F can grant multiple simultaneous operations on IoT devices through just one user authentication. We implement a prototype to further evaluate the performance of G2F. Based on our realization on the commercial IoT server, i.e., Alibaba Cloud, G2F demonstrates the ability to protect against malicious attacks with high authentication efficiency.

Speaker verification system has gained great popularity in recent years, especially with the development of deep neural networks and Internet of Things. However, the security of speaker verification system based on deep neural networks has not been well investigated. In this paper, we propose an attack to spoof the state-of-the-art speaker verification system based on generalized end-to-end (GE2E) loss function for misclassifying illegal users into the authentic user. Specifically, we design a novel loss function to deploy a generator for generating effective adversarial examples with slight perturbation and then spoof the system with these adversarial examples to achieve our goals. The success rate of our attack can reach 82% when cosine similarity is adopted to deploy the deep-learning-based speaker verification system. Beyond that, our experiments also reported the signal-to-noise ratio at 76 dB, which proves that our attack has higher imperceptibility than previous works. In summary, the results show that our attack not only can spoof the state-of-the-art neural-network-based speaker verification system but also more importantly has the ability to hide from human hearing or machine discrimination.

In Internet of Things, where billions of devices with limited resources are communicating with each other, security has become a major stumbling block affecting the progress of this technology. Existing authentication schemes based on digital signatures have overhead costs associated with them in terms of computation time, battery power, bandwidth, memory, and related hardware costs. Radio frequency fingerprint (RFF), utilizing the unique device-based information, can be a promising solution for IoT. However, traditional RFFs have become obsolete because of low reliability and reduced user capability. Our proposed solution, Metasurface RF-Fingerprinting Injection (MeRFFI), is to inject a carefully-designed radio frequency fingerprint into the wireless physical layer that can increase the security of a stationary IoT device with minimal overhead. The injection of fingerprint is implemented using a low cost metasurface developed and fabricated in our lab, which is designed to make small but detectable perturbations in the specific frequency band in which the IoT devices are communicating. We have conducted comprehensive system evaluations including distance, orientation, multiple channels where the feasibility, effectiveness, and reliability of these fingerprints are validated. The proposed MeRFFI system can be easily integrated into the existing authentication schemes. The security vulnerabilities are analyzed for some of the most threatening wireless physical layer-based attacks.

Multi-factor user authentication (MFUA) becomes increasingly popular due to its superior security comparing with single-factor user authentication. However, existing MFUAs require multiple interactions between users and different authentication components when sensing the multiple factors, leading to extra overhead and bad use experiences. In this paper, we propose a secure and user-friendly MFUA system, namely BioDraw, which utilizes four categories of biometrics (impedance, geometry, composition, and behavior) of human hand plus the pattern-based password to identify and authenticate users. A user only needs to draw a pattern on a RFID tag array, while four biometrics can be simultaneously collected. Particularly, we design a gradient-based pattern recognition algorithm for pattern recognition and then a CNN-LSTM-based classifier for user recognition. Furthermore, to guarantee the systemic security, we propose a novel anti-spoofing scheme, called Binary ALOHA, which utilizes the inhabit randomness of RFID systems. We perform extensive experiments over 21 volunteers. The experiment result demonstrates that BioDraw can achieve a high authentication accuracy (with a false reject rate less than 2%) and is effective in defending against various attacks.

Internet of Things (IoT) devices are hindered from communicating with their neighbors by incompatible protocols or electromagnetic interference. Existing solutions adopting physical covert channels have limitations in receiver distinction, additional hardware, conditional placement, or physical contact. Our system, Deaf-Aid, utilizes the stealthy speaker-to-gyroscope channel to build robust protocol-independent communication with automatic receiver identification. Deaf-Aid exploits ultrasonic signals at a frequency corresponding to the target receiver, forcing the gyroscope inside to resonate, so as to convey information. We probe the relationship among axes in a gyroscope to surmount frequency offset ingeniously and support multi-channel communication. Meanwhile, Deaf-Aid identifies the receivers automatically via device fingerprints constituted by the diversity of resonant frequency ranges. Furthermore, we entitle Deaf-Aid the capability of mobile communication which is an essential demand for IoT devices. We address the challenge of accurate signals recovery from motion interference. Extensive evaluations demonstrate that Deaf-Aid yields 47bps with BER lower than 1% under motion interference. To our best knowledge, Deaf-Aid is the first work to enable stealthy mobile IoT communication on the basis of inertial motion sensors.

The advent of smart devices has caused unprecedented security and privacy concerns to its users. Although the fingerprint technology is a go-to biometric solution in high-impact applications (e.g., smart-phone security, monetary transactions and international-border verification), the existing fingerprint scanners are vulnerable to spoofing attacks via fake-finger and cannot be employed across smart devices (e.g., wearables) due to hardware constraints. We propose SonicPrint that extends fingerprint identification beyond smartphones to any smart device without the need for traditional fingerprint scanners. SonicPrint builds on the fingerprint-induced sonic effect (FiSe) caused by a user swiping his fingertip on smart devices and the resulting property, i.e., different users' fingerprint would result in distinct FiSe. As the first exploratory study, extensive experiments verify the above property with 31 participants over four different swipe actions on five different types of smart devices with even partial fingerprints. SonicPrint achieves up to a 98% identification accuracy on smartphone and an equal-error-rate (EER) less than 3% for smartwatch and headphones. We also examine and demonstrate the resilience of SonicPrint against fingerprint phantoms and replay attacks. A key advantage of SonicPrint is that it leverages the already existing microphones in smart devices, requiring no hardware modifications. Compared with other biometrics including physiological patterns and passive sensing, SonicPrint is a low-cost, privacy-oriented and secure approach to identify users across smart devices of unique form-factors.

With the continuing growth of voice-controlled devices, voice metrics have been widely used for user identification. However, voice biometrics is vulnerable to replay attacks and ambient noise. We identify that the fundamental vulnerability in voice biometrics is rooted in its indirect sensing modality (e.g., microphone). In this paper, we present VocalPrint, a resilient mmWave interrogation system which directly captures and analyzes the vocal vibrations for user authentication. Specifically, VocalPrint exploits the unique disturbance of the skin-reflect radio frequency (RF) signals around the near-throat region of the user, caused by the vocal vibrations during communication. The complex ambient noise is isolated from the RF signal using a novel resilience-aware clutter suppression approach for preserving fine-grained vocal biometric properties. Afterward, we extract the text-independent vocal tract and vocal source features and input them to an ensemble classifier for user authentication. VocalPrint is practical as it leverages a low-cost, portable, and energy-efficient hardware allowing effortless transition to a smartphone while having sufficient usability as typical voice authentication systems due to its non-contact nature. Our experimental results from 41 participants with different interrogation distances, orientations, and body motions show that VocalPrint can achieve over 96% authentication accuracy even under unfavorable conditions. We demonstrate the resilience of our system against complex noise interference and spoof attacks of various threat levels.

Biometric authentication offers advantages over current security practices. Unlike keys and tokens, biometrics are never lost or stolen. Unlike passwords, biometrics cannot be forgotten. However, existing biometric systems are with controversy: once divulged, they are compromised forever. To this end, this paper explores a truly cancelable brain-based biometric system for the first time. Specifically, we present a new psychophysiological protocol via non-volitional brain response for trustworthy user authentication, with an application example of smart headwear. More specifically, we address the following research challenges in a theoretical and experimental combined manner: (1) how to generate reliable brain responses with sophisticated visual stimuli; (2) how to acquire effective brain response and analyze unique features in them for authentication; and (3) how to reset and change brain biometrics when the current biometric credential is divulged. To evaluate the performance of the proposed system, we conducted a pilot study and achieved an f-score accuracy of 95.46 percent and equal error rate (EER) of 2.503 percent, thereby demonstrating the potential feasibility of neurofeedback based biometrics for smart headwear applications. Further, the cancelability study proves the effectiveness of the reset brain password. To the best of our knowledge, it is the first in-depth research study on truly cancelable brain biometrics.

Wi-Fi signals-based person identification attracts increasing attention in the booming Internet-of-Things era mainly due to its pervasiveness and passiveness. Most previous work applies gaits extracted from WiFi distortions caused by person walking to achieve the identification. However, to extract useful gait, a person must walk along a pre-defined path for several meters, which requires user high collaboration and increases identification time overhead, thus limiting use scenarios. Moreover, gait based work has severe shortcoming in identification performance, especially when the user volume is large. In order to eliminate above limitations, in this paper, we present an operation-free person identification system, namely WiPIN, that requires least user collaboration and achieves good performance. WiPIN is based on an entirely new insight that Wi-Fi signals would carry person body information when propagating through the body, which is potentially discriminated for person identification. Then we demonstrate the feasibility on commodity off-the-shelf Wi-Fi devices by well- designed signal pre-processing, feature extraction, and identity matching algorithms. Results show that WiPIN achieves 92% identification accuracy over 30 users, high robustness to various experimental settings, and low identifying time overhead, i.e., less than 300ms.

Hidden electronics possess the risk of both security threat and privacy intrusion. We present a wireless hidden electronic recognition system, through electronic components unique mmWave nonlinear responses to identify the threats. We then evaluate E-Eye's performance and robustness with a controlled experiment and a field study using iconic devices and score the system with metrics. Results prove that E-Eye is an accurate and robust hidden electronic recognition system.

With the growth of the elder population, fall risk evaluation is crucial to prevent elders from serious injuries, as well as reduce related financial burdens. A balance assessment, timed up and go (TUG), has been widely used to estimate fall risk. The standardized TUG focuses on flat ground walking with no environmental variance. Therefore, it falls short of assessing an individual's gait adaptability. Being able to adjust steps in response to environmental changes, for example, needing to navigate around or over a child's toy left on the sidewalk, is essential to avoid fall risk and fundamental to community ambulation. To this end, we propose four environment-adapting TUGs designed to assess one's ability to adapt gait in complex environments and a compatible system named Smart Insole TUG (SITUG), which provides real-time, feature-rich, and ease-of-operation TUG analysis. Based on experimental results, SITUG is capable of extracting gait related spatial-temporal features with all mean accuracies over 92%. Besides, the system achieves a mean accuracy of 92.23% in segmenting five TUG phases.

The rapid development of Internet-of-Things is yielding a huge volume of time series data, the real-time mining of which becomes a major load for data centers. The computation bottleneck in time series data mining is distance function, which is the fundamental element of many high data mining tasks. Recently various software optimization and hardware acceleration techniques have been proposed to tackle the challenge. However, each of these techniques is only designed or optimized for a specific distance function. To address this problem, in this paper we propose MDA, a high-throughput reconfigurable memristor-based distance accelerator for real-time and energy-efficient data mining with time series in data centers. Common circuit structure is extracted for efficiency, and the circuit can be configured to any specific distance functions. Particularly, we adopt the emerging device memristor for the design of MDA. Comprehensive experiments are presented with public available datasets to evaluate the performance of the proposed MDA. Experimental results show that compared with existing works, MDA has achieved a speedup of 3.5×-376× on performance and an improvement of 1-3 orders of magnitude on energy efficiency with little accuracy loss.

Motor learning is the ability to acquire a new motor skill, which plays an important role in rehabilitation as patients learn exercise programs or modify movements to regain pain free function. In this paper, we design an easy-to-use multimodal sensor system to assess motor learning. We developed a motor learning assessment device with a touch screen and Leap Motion to record the subject hand movement during a Serial Reaction Time Task(SRTT). The SRTT consists of upper limb reaching to targets in multi-dimensions. The device records metrics of time and movement efficiency and examines motor learning based on data analysis. This device can provide clinicians with data that can inform their approach to training. We recruited a total of 11 participants, with and without chronic pain to evaluate the device using a classifier model to assess participants' performance. The model shows our system works well to identify motor learning differences in individuals with and without chronic pain.

Stroke is a leading cause of long-term impairment, causing a fatality if not act upon in time. Home-based post-stroke rehabilitation plays an important role in helping patients to regain normal mobility and functionality at their residence. However, existing home-based rehabilitation approaches fail to effectively motivate patients on frequent engagement with exercise to achieve the intended outcome. In this paper, we develop VRInsole, a synthetical solution combining a Smart Insole footwear sensor and virtual reality (VR), targeting lower extremity mobility training in an immersive environment for stroke rehabilitation. Specifically, the motion information collected from the Smart Insole serve as the input for the VR to perform corresponding exercise animations. To prove the feasibility of VRInsole, an experiment is conducted on the recognition of lower extremity motion direction, which achieves an average accuracy of 93.9%.

Motor impairment is common following stroke. Diminished strength and coordination contribute to reduced ability to perform activities of daily living. The existing healthcare models focus on delivering rehabilitation during the first few months following stroke. Yet, to regain motor control to the greatest degree, rehabilitation should continue across the lifespan. Currently, individuals with stroke are responsible for self-managing their rehabilitation once therapist guided rehabilitation has concluded. Individuals with stroke are frequently given a written home exercise program to help guide their home rehabilitation, but poor compliance demonstrates a better approach is necessary. In this study, we propose BiGRA, a novel system to more effectively facilitate in-home bilateral rehabilitation. This system holds merits in: (1) An end-to-end task-oriented system for bilateral grip control which emphasizes the modulation of grip coordination between hands; and (2) Innovative metrics framework to quantitatively analyze the motor control performance. The evaluation shows that BiGRA can objectively measure the patients' task performance and is a promising assessment tool for stroke rehabilitation.

In recent years, biometric techniques (e.g., fingerprint or iris) are increasingly integrated into mobile devices to offer security advantages over traditional practices (e.g., passwords and PINs) due to their ease of use in user authentication. However, existing biometric systems are with controversy: once divulged, they are compromised forever - no one can grow a new fingerprint or iris. This work explores a truly cancelable brain-based biometric system for mobile platforms (e.g., smart headwear). Specifically, we present a new psychophysiological protocol via non-volitional brain response for trustworthy mobile authentication, with an application example of smart headwear. Particularly, we address the following research challenges in mobile biometrics with a theoretical and empirical combined manner: (1) how to generate reliable brain responses with sophisticated visual stimuli; (2) how to acquire the distinct brain response and analyze unique features in the mobile platform; (3) how to reset and change brain biometrics when the current biometric credential is divulged. To evaluate the proposed solution, we conducted a pilot study and achieved an f -score accuracy of 95.46% and equal error rate (EER) of 2.503%, thereby demonstrating the potential feasibility of neurofeedback based biometrics for smart headwear. Furthermore, we perform the cancelability study and the longitudinal study, respectively, to show the effectiveness and usability of our new proposed mobile biometric system. To the best of our knowledge, it is the first in-depth research study on truly cancelable brain biometrics for secure mobile authentication.

The rapid development of Internet of Things is yielding a huge volume of time series data, the real-time mining of which becomes a major load for data centers. The computation bottleneck in time series mining is the distance measure, in which dynamic time warping (DTW) is one of the most widely used distance measures. Recently, various software optimization and hardware acceleration techniques have been proposed for DTW acceleration. However, the throughput and energy efficiency of DTW are still big concerns considering the ever-increasing volume of times series. In this paper, we propose a high-throughput and efficient memristor-based DTW architecture for real-time time series mining on data centers. Specifically, memristors have been adopted for both computation and configuration of the computing architecture. The computation flow in this architecture is fully presented in a continuous and asynchronous manner. To improve the computation efficiency, we propose an early lower bound algorithm by exploiting the predictability in the circuit characteristic. Experiments are performed with module evaluation and end-to-end evaluation including three popular applications: 1) similarity search; 2) classification; and 3) anomaly detection. Experimental results indicate that, compared to existing approaches, the speedup and energy efficiency improvement are 12x-43x and 51x-287x, respectively.

As a micro-engineered biomimetic system to replicate key functions of living organs, organ-on-a-chip (OC) technology provides a high-throughput model for investigating complex cell interactions with both high temporal and spatial resolutions in biological studies. Typically, microscopy and high-speed video cameras are used for data acquisition, which are expensive and bulky. Recently, compressed sensing (CS) has increasingly attracted attentions due to its extremely low-complexity structure and low sampling rate. However, there is no CS solution tailored for tempo-spatial information acquisition. In this paper, we propose tempo-spatial CS (TS-CS), a unified CS architecture for OC stream, which achieves significant cost reduction and truly combines sensing with compression along the temporal and spatial domains. We point out that TS-CS can consistently achieve better performance by exploiting tempo-spatial compressibility in OC data. To this end, we comprehensively evaluate the system performance by employing four different bases for CS. With comparison to the traditional way, we show that TS-CS always obtains better recovery result with a throughput bound and can achieve around throughput improvement under a reconstruction demand by applying discrete cosine transform matrix as the basis.

Continuous authentication is of great importance to maintain the security level of a system throughout the login session. The goal of this work is to investigate a trustworthy, continuous, and non-contact user authentication approach based on a heart-related biometric that works in a daily-life environment. To this end, we present a novel, continuous authentication system, namely Cardiac Scan, based on geometric and non-volitional features of the cardiac motion. Cardiac motion is an automatic heart deformation caused by self-excitement of the cardiac muscle, which is unique to each user and is difficult (if not impossible) to counterfeit. Cardiac Scan features intrinsic liveness detection, unobtrusiveness, cost-effectiveness, and high usability. We prototype a remote, high-resolution cardiac motion sensing system based on the smart DC-coupled continuous-wave radar. Fiducial-based invariant identity descriptors of cardiac motion are extracted after the radar signal demodulation. We conduct a pilot study with 78 subjects to evaluate Cardiac Scan in accuracy, authentication time, permanence, evaluation in complex conditions, and vulnerability. Specifically, Cardiac Scan achieves 98.61% balanced accuracy (BAC) and 4.42% equal error rate (EER) in a real-world setup. We demonstrate that Cardiac Scan is a robust and usable continuous authentication system.

Nurses regularly perform patient handling activities. These activities with awkward postures expose healthcare providers to a high risk of overexertion injury. The recognition of patient handling activities is the first step to reduce injury risk for caregivers. The current practice on workplace activity recognition is based on human observational approach, which is neither accurate nor projectable to a large population. In this paper, we aim at addressing these challenges. Our solution comprises a smart wearable device and a novel spatio-temporal warping (STW) pattern recognition framework. The wearable device, named Smart Insole 2.0, is equipped with a rich set of sensors and can provide an unobtrusive way to automatically capture the information of patient handling activities. The STW pattern recognition framework fully exploits the spatial and temporal characteristics of plantar pressure by calculating a novel warped spatio-temporal distance, to quantify the similarity for the purpose of activity recognition. To validate the effectiveness of our framework, we perform a pilot study with eight subjects, including eight common activities in a nursing room. The experimental results show the overall classification accuracy achieves 91.7%. Meanwhile, the qualitative profile and load level can also be classified with accuracies of 98.3% and 92.5%, respectively.

Additive manufacturing, also known as 3D printing, has been increasingly applied to fabricate highly intellectual property (IP) sensitive products. However, the related IP protection issues in 3D printers are still largely underexplored. On the other hand, smartphones are equipped with rich onboard sensors and have been applied to pervasive mobile surveillance in many applications. These facts raise one critical question: is it possible that smartphones access the side-channel signals of 3D printer and then hack the IP information? To answer this, we perform an end-to-end study on exploring smartphone-based side-channel attacks against 3D printers. Specifically, we formulate the problem of the IP side-channel attack in 3D printing. Then, we investigate the possible acoustic and magnetic side-channel attacks using the smartphone built-in sensors. Moreover, we explore a magnetic-enhanced side-channel attack model to accurately deduce the vital directional operations of 3D printer. Experimental results show that by exploiting the side-channel signals collected by smartphones, we can successfully reconstruct the physical prints and their G-code with Mean Tendency Error of 5.87% on regular designs and 9.67% on complex designs, respectively. Our study demonstrates this new and practical smartphone-based side channel attack on compromising IP information during 3D printing.

Gait analysis is an important medical diagnostic process and has many applications in healthcare, rehabilitation, therapy, and exercise training. However, typical gait analysis has to be performed in a gait laboratory, which is inaccessible for a large population and cannot provide natural gait measures. In this paper, we present a novel sensor device, namely, Smart Insole, to tackle the challenge of efficient gait monitoring in real life. An array of electronic textile (eTextile)-based pressure sensors are integrated in the insole to fully measure the plantar pressure. Smart Insole is also equipped with a low-cost inertial measurement unit including a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer to capture the gait characteristics in motion. Smart Insole can offer precise acquisition of gait information. Meanwhile, it is lightweight, thin, and comfortable to wear, providing an unobtrusive way to perform the gait monitoring. Furthermore, a smartphone graphic user interface is developed to display the sensor data in real-time via Bluetooth low energy. We perform a set of experiments in four real-life scenes including hallway walking, ascending/descending stairs, and slope walking, where gait parameters and features are extracted. Finally, the limitation and improvement, wearability and usability, further work, and healthcare-related potential applications are discussed.

The past decades have witnessed a rapid surge in new sensing and monitoring devices for well-being and healthcare. One key representative in this field is body sensor networks (BSNs). However, with advances in sensing technologies and embedded systems, wireless communication has gradually become one of the dominant energy-consuming sectors in BSN applications. Recently, compressed sensing (CS) has attracted increasing attention in solving this problem due to its enabled sub-Nyquest sampling rate. In this paper, we investigate the quantization effect in CS architecture and argue that the quantization configuration is a critical factor of the energy efficiency for the entire CS architecture. To this end, we present a novel configurable quantized compressed sensing (QCS) architecture, in which the sampling rate and quantization are jointly explored for better energy efficiency. Furthermore, to combat the computational complexity of the configuration procedure, we propose a rapid configuration algorithm, called RapQCS. According to the experiments involving several categories of real biosignals, the proposed configurable QCS architecture can gain more than 66% performance-energy tradeoff than the fixed QCS architecture. Moreover, our proposed RapQCS algorithm can achieve over 150× speedup on average, while decreasing the reconstructed signal fidelity by only 2.32%.

Sleep monitoring is receiving increased attention in the healthcare community, because the quality of sleep has a great impact on human health. Existing in-home sleep monitoring devices are either obtrusive to the user or cannot provide adequate sleep information. To this end, we present SleepSense, a contactless and low-cost sleep monitoring system for home use that can continuously detect the sleep event. Specifically, SleepSense consists of three parts: an electromagnetic probe, a robust automated radar demodulation module, and a signal processing framework for sleep event recognition, including on-bed movement, bed exit, and breathing event. We present a prototype of the SleepSense system, and perform a set of comprehensive experiments to evaluate the performance of sleep monitoring. Using a real-case evaluation, experimental results indicate that SleepSense can perform effective sleep event detection and recognition in practice.