In real-time human-machine interaction (HMI) applications, hand gesture recognition (HGR) requires high accuracy with low latency. Surface electromyography (sEMG), a physiological electrical signal reflecting muscle activation, is extensively used in HMI. Recently, transient sEMG, generated during the gesture transitions, has been employed in HGR to achieve lower observational latency compared to steady-state sEMG. However, the use of long feature windows (up to 200 ms) still make it less desirable in low-latency HMI. In addition, most studies have relied on remote computing, where remote data processing and large data transfer result in high computation and network latency. In this paper, we proposed a method leveraging transient high density sEMG (HD-sEMG) and in-sensor computing to achieve low-latency HGR. An sEMG contrastive convolution network (sCCN) was proposed for HGR. The mean absolute value and its average integration were used to train the sCCN in a contrastive learning manner. In addition, all signal acquisition, data processing, and pattern recognition processes were deployed within designed sensor for in-sensor computing. Compared to the state-of-the-art study using multi-channel 200-ms transient sEMG, our proposed method achieved a comparable HGR accuracy of 0.963, and a 58% lower observational latency of only 84 ms. In-sensor computing realizes a 4 times lower computation latency of 3 ms, and significantly reduces the network latency to 2 ms. The proposed method offers a promising approach to achieving low-latency HGR without compromising accuracy. This facilitates real-time HMI in biomedical applications such as prostheses, exoskeletons, virtual reality, and video games.

Rehabilitation robots offer promising clinical benefits, however, a critical concern is whether patients can perform exercises correctly without therapist guidance. Patients with stroke, in particular, often resort to compensatory movements without supervision, leading to less-than-optimal outcomes. To address this issue, our work aims to detect and minimize the use of compensation during robotic rehabilitation. Specifically, we developed an artificial neural network (ANN) that uses pressure distribution data to detect compensatory motions. Furthermore, a rehabilitation robot provided real-time force feedback to stroke patients to reduce trunk compensation when necessary. Our experiment involved 8 stroke patients and 12 healthy individuals who completed reaching tasks using an upper limb rehabilitation robot. Results demonstrated the effectiveness of our approach in both offline and online compensation detection. Moreover, we observed a significant reduction in trunk compensation among stroke patients. This study provides a promising solution for unsupervised robotic rehabilitation for stroke patients, paving the way for improved rehabilitation outcomes.

Pectus excavatum (PE) is one of the most common congenital sternal deformities. Accurate preoperative diagnosis of PE is of great significance for subsequent correction and improvement of the patient's quality of life. However, current diagnostic methods rely on the calculation of some PE indices, which is a heavy workload for physical therapists and suffers from measurement errors. To address this issue, we propose an end-to-end automatic assessment of PE, which features a cascaded structure of CNN and LSTM. Specifically, the high-level feature representations of CT images are extracted by the pretrained CNN and then processed through the LSTM layers for classification. In addition, we build up a medical image dataset for PE diagnosis by collecting chest CT images of 42 subjects. Results on this dataset show that the proposed CNN-LSTM framework achieves a relatively high accuracy of 90.20%, which provides a new perspective for the automatic diagnosis of PE in clinics.

Decoding auditory attention in a cocktail party from neural activities is crucial in the brain-computer interfaces (BCIs). Given that the speech-electroencephalography (EEG) relationships are informative about attentional focus, we propose a novel framework called the biologically inspired attention network (BIAnet) to capture the interactions between EEG and speech. With the neural attention mechanism, the BIAnet can model how each EEG frequency band is related to the subband envelopes of speech by dynamically assigning weights to individual frequency bands at run-time. Results show that the proposed BIAnet outperforms state-of-the-art AAD methods on two publicly available datasets. We also analyze how the BIAnet works and the frequency-specific interactions between EEG and speech signals through data visualization. Overall, the proposed BIAnet provides an accurate, low-latency, and interpretable AAD approach, which has the potential to be extended to general problems in BCIs.

Humans have the ability to focus on one of the sound sources in a noisy scene, which is critical for everyday communication. Auditory attention detection (AAD) seeks to detect selective attention from one’s brain signals. For AAD to be useful in brain–computer interface applications, new approaches with low computational cost, high classification performance, and low latency are required to be developed. In this study, we proposed a novel neural-inspired architecture to mimic the neural computation and coding strategy in the brain for electroencephalography-based AAD. We validated our model through data visualization, and conducted experiments on two publicly available databases. For both KUL and DTU databases, it outperforms both linear and convolutional neural network (CNN) models with consistent improvements from 1 s to 5 s decision windows in terms of detection accuracy. Although the accuracy of the proposed neural-inspired model is inferior to the state-of-the-art spatio-spectral feature (SSF)-CNN model, the computational cost of our model is less than 1% of SSF-CNN’s. Moreover, the neural-inspired decoder is more hardware friendly and energy-efficient due to its biological computing scheme. Overall, the proposed neural-inspired architecture realizes a fast, accurate, and low energy expenditure AAD, which is a big step forward towards practical neuro-steered hearing aids.

Humans have the ability to pay attention to one of the sound sources in a multispeaker acoustic environment. Auditory attention detection (AAD) seeks to detect the attended speaker from one’s brain signals that will enable many innovative human–machine systems. However, effective representation learning of electroencephalography (EEG) signals remains a challenge. In this article, we propose a neural attention mechanism that dynamically assigns differentiated weights to the subbands and the channels of EEG signals to derive discriminative representations for AAD. In the nutshell, we would like to build a computational attention mechanism, i.e., neural attention, to model the auditory attention in human brain. We incorporate the proposed neural attention into an AAD system, and validate the neural attention mechanism through comprehensive experiments on two publicly available datasets. The experimental results demonstrate that the proposed system significantly outperforms the state-of-the-art reference baselines.

OBJECTIVES: Compensations are commonly employed by patients with stroke during rehabilitation without therapist supervision, leading to suboptimal recovery outcomes. This study investigated the feasibility of the real-time monitoring of compensation in patients with stroke by using pressure distribution data and machine learning algorithms. Whether trunk compensation can be reduced by combining the online detection of compensation and haptic feedback of a rehabilitation robot was also investigated. METHODS: Six patients with stroke did three forms of reaching movements while pressure distribution data were recorded as Dataset1. A support vector machine (SVM) classifier was trained with features extracted from Dataset1. Then, two other patients with stroke performed reaching tasks, and the SVM classifier trained by Dataset1 was employed to classify the compensatory patterns online. Based on the real-time monitoring of compensation, a rehabilitation robot provided an assistive force to patients with stroke to reduce compensations. RESULTS: Good classification performance (F1 score > 0.95) was obtained in both offline and online compensation analysis using the SVM classifier and pressure distribution data of patients with stroke. Based on the real-time detection of compensatory patterns, the angles of trunk rotation, trunk lean-forward and trunk-scapula elevation decreased by 46.95%, 32.35% and 23.75%, respectively. CONCLUSION: High classification accuracies verified the feasibility of detecting compensation in patients with stroke based on pressure distribution data. Since the validity and reliability of the online detection of compensation has been verified, this classifier can be incorporated into a rehabilitation robot to reduce trunk compensations in patients with stroke.