Socially assistive robots (SARs) can act as caregivers and use touch gestures to interact with people such as children with autism. However, designing a touch perception system that can reliably detect these gestures remains a challenge. To address this gap, we replicated a Do-It-Yourself (DIY) skin-like sensor and evaluated its potential to identify eight social touch gestures: Fistbump, Hitting, Holding, Poking, Squeezing, Stroking, Tapping, and Tickling. Our sensor design builds on a recent silicone-based sensor to collect spatiotemporal gesture data and a load cell to capture force information. We collected touch gestures from 20 adults in a user study. Then, we built a touch perception algorithm with a Convolutional Neural Network-Long Short-Term Memory (CNN- LSTM) model, achieving 94% in gesture classification accuracy with to-fold validation and 69 % accuracy with subject-dependent splitting with the combined touch and force data.

Electromyography (EMG) signals have been extensively used to capture facial muscle movements while speaking since they are one of the most closely related bio-signals generated during speech production. In this work, we focus on speech acoustics to EMG prediction. We present a comparative study of ten different EMG signal-based features including Time Domain (TD) features existing in the literature to examine their effectiveness in speech acoustics to EMG inverse (AEI) mapping. We propose a novel feature based on the Hilbert envelope of the filtered EMG signal. The raw EMG signal is reconstructed from these features as well. For the AEI mapping, we use a bi-directional long short-term memory (BLSTM) network in a session-dependent manner. To estimate the raw EMG signal from the EMG features, we use a CNN-BLSTM model comprising of a convolution neural network (CNN) followed by BLSTM layers. AEI mapping performance using the BLSTM network reveals that the Hilbert envelope based feature is predicted from speech with the highest accuracy, among all the features. Therefore, it could be the most representative feature of the underlying muscle activation during speech production. The proposed Hilbert envelope feature, when used together with the existing TD features, improves the raw EMG signal reconstruction performance compared to using the TD features alone. Copyright © 2021 ISCA.

We present a novel approach to generating fundamental frequency (intonation and voicing) trajectories in an EMG-to-Speech conversion Silent Speech Interface, based on quantizing the EMG-to-F0mappings target values and thus turning a regression problem into a recognition problem. We present this method and evaluate its performance with regard to the accuracy of the voicing information obtained as well as the performance in generating plausible intonation trajectories within voiced sections of the signal. To this end, we also present a new measure for overall F0trajectory plausibility, the trajectory-label accuracy (TLAcc), and compare it with human evaluations. Our new F0generation method achieves a significantly better performance than a baseline approach in terms of voicing accuracy, correlation of voiced sections, trajectory-label accuracy and, most importantly, human evaluations.