Deepfakes, an increasingly prevalent form of information attack, pose serious threats to security and privacy. Deepfake voice attacks, in particular, have the potential to cause widespread disruption, creating an urgent need for an effective detection system. In this research, we propose the Trident of Poseidon - a novel set of triad training strategies aimed at enhancing the generalizability of deepfake voice detection models. Our solution comprises three key components: (1) Supervised Contrastive Learning, (2) Hard Negative Mining by Audio Re-synthesizing, and (3) Effective Proactive Batch Sampling. Together, these enable the model to learn more robust features. Our extensive experiments demonstrate that our approach outperforms existing methods in both in-domain and out-of-domain testing scenarios, making significant strides toward securing digital media against deepfake voice attacks.

Multi-limb coordination is necessary for many daily activities. It also has particular relevance for a growing area of human-system applications such as teleoperated robot operation and supernumerary robot operation where an additional robotic limb may complement natural limb functions. Introducing sup-plemental sensory information, specifically haptic feedback, has shown the potential to improve assistive robotics control, human-machine interactions, and even human motor learning. However, there has been little research on how sensory information, including haptic feedback and visual information, contributes to the execution of multi-limb coordinated tasks. In this paper, we designed a tri-limb coordinated task where participants move a cursor to a randomly positioned target on a screen guided by variations of both haptic feedback and visual information. Our results indicated that while haptic feedback could partially sub-stitute for visual information, it could not completely replace nor surpass the task performance achieved with visual information as feedback. Importantly, haptic feedback does not negatively impact performance metrics such as how often the participant coordinates multiple limbs simultaneously (coordination score), controls their limbs without redundant movement (normalized sub-optimal path), completes the task (success rate), and the mental workload perceived during the task (cognitive load). This study provides insight into the contribution of sensory feedback in facilitating multi-limb coordination, providing relevant guidance for numerous applications of human interactions with assistive technologies.

Over the past decade, there is a growing interest in collaborative learning that can enhance AI models of multiple parties. However, it is still challenging to enhance performance them without sharing private data and models from individual parties. One recent promising approach is to develop distillation-based algorithms that exploit unlabeled public data but the results are still unsatisfactory in both theory and practice. To tackle this problem, we rigorously analyze a representative distillation-based algorithm in the view of kernel regression. This work provides the first theoretical results to prove the (nearly) minimax optimality of the nonparametric collaborative learning algorithm that does not directly share local data or models in massively distributed statistically heterogeneous environments. Inspired by our theoretical results, we also propose a practical distillation-based collaborative learning algorithm based on neural network architecture. Our algorithm successfully bridges the gap between our theoretical assumptions and practical settings with neural networks through feature kernel matching. We simulate various regression tasks to verify our theory and demonstrate the practical feasibility of our proposed algorithm.

Voice phishing (vishing) is increasingly popular due to the development of speech synthesis technology. In particular, the use of deep learning to generate an arbitrary-content audio clip simulating the victim’s voice makes it difficult not only for humans but also for automatic speaker verification (ASV) systems to distinguish. Countermeasure (CM) systems have been developed recently to help ASV combat synthetic speech. In this work, we propose BTS-E, a framework to evaluate the correlation between Breathing, Talking (speech), and Silence sounds in an audio clip, then use this information for deepfake detection tasks. We argue that natural human sounds, such as breathing, are hard to synthesize by Text-to-speech (TTS) system. We conducted a large-scale evaluation using ASVspoof 2019 and 2021 evaluation set to validate our hypothesis. The experiment results show the applicability of the breathing sound feature in detecting deepfake voices. In general, the proposed system significantly increases the performance of the classifier by up to 46%.

Federated Learning (FL) is a collaborative machine learning paradigm for data privacy preservation. Recently, a knowledge distillation (KD) based information sharing approach in FL, which conducts ensemble distillation on an unlabeled public dataset, has been proposed. However, despite its experimental success and usefulness, the theoretical analysis of the KD based approach has not been satisfactorily conducted. In this work, we build a theoretical foundation of the ensemble distillation framework in federated learning from the perspective of kernel ridge regression (KRR). In this end, we propose a KD based FL algorithm for KRR models which is related with some existing KD based FL algorithms, and analyze our algorithm theoretically. We show that our algorithm makes local prediction models as much powerful as the centralized KRR model (which is a KRR model trained by all of local datasets) in terms of the convergence rate of the generalization error if the unlabeled public dataset is sufficiently large. We also provide experimental results to verify our theoretical results on ensemble distillation in federated learning.

Embodiment is the experience of owning and controlling our bodies. It is a product of cognitive sensorimotor integration processes and by manipulating what we see and feel, we can extend the perceived boundaries of our bodies to include non-biological objects. Consequently, it is of interest to many human-robotic control applications that aim to promote the seamless interaction of the human user and machine. The sense of embodiment has three constituent components that robotic systems may engage with: the Sense of Ownership (SoO), the Sense of Agency (SoA), and our peripersonal space (PPS). Despite the recognitionthat each is linked to the other, how these components interact to form a cohesive sense of embodiment remains poorly understood. To address this issue, in this work, we designed an embodiment research platform that allows us to explore the relationships between the SoO, SoA, and PPS. We developed a sensorimotor interface, experimental setup, and protocol in which a robotic hand mirrored the actions of the participants’ real hand. Participants performed a series of grasping tasks in which we manipulated the cutaneous sensations they experienced and the position of the robot relative to their body. We then employed multiple independent measures of SoO, SoA, and PPS. Our results indicated that (1) cutaneous feedback improved all three components of embodiment, and (2) there appeared to be multiple complex interrelationships among the three components and their measures.