Remote photoplethysmography (rPPG) for heart rate (HR) measurement based on facial videos has recently attracted increasing attention. However, most existing methods focus on average heart rate (AHR) over a period rather than instantaneous heart rate (IHR), which better reflects physical and mental states. To address this issue, we propose a novel rPPG-based method for measuring IHR values from facial videos. Our method employs the wavelet synchrosqueezed transform (WSST) to generate time-frequency representations (TFRs) of chrominance (CHROM) signals from multiple facial regions of interest (ROIs), synchronously reflecting the IHR during a video segment. Furthermore, the TransUNet is introduced to refine these TFR images, enhancing the ridge line information related to IHRs. Comprehensive comparisons and ablation studies on four public datasets (UBFC-rPPG, PURE, UBFC-Phys, and MMPD) reveal that our WSST-UNet method achieves superior performance over several typical rPPG methods, achieving mean absolute errors (MAE) of 2.34 beats per minute (bpm), 1.29 bpm, 5.03 bpm, and 6.58 bpm, respectively. The proposed method offers a promising solution for practical application in video-based IHR measurements.

While patient-specific seizure prediction deep learning (DL) models can deliver remarkable performance tailored to individual patients, the development of patient-independent models that offer satisfactory cross-subject performance holds greater significance and practicality. However, these patient-independent models, which leverage electroencephalogram (EEG) data from multiple patients, may give rise to privacy concerns. This is because EEG data contains sensitive information regarding individuals’ health and mental states. Consequently, from a privacy-preserving perspective, patients may desire the removal of their data information from trained models. Yet, accommodating such forgetting requests presents a formidable challenge: how to enable DL models to forget the data information of specific patients without compromising the performance for others. Although retraining a model from scratch without the data of a specific patient can somewhat address this issue, it becomes computationally prohibitive, especially with large datasets. To tackle this, we introduce an efficient machine unlearning approach called feature unlearning (FU) for seizure prediction. This method modifies the feature projection distribution of specific patients’ data within trained models to match that of models retrained from scratch. Our proposed FU method comprises two primary components: 1) feature shifting, which alters the original distribution of feature projection of specific patients in trained models and feature retaining, which mitigates the adverse effects of feature shifting on other patients, preserving their overall performance through knowledge distillation. We assess our FU method using the CHB-MIT dataset. The results demonstrate that our FU approach can effectively remove the data information of specific patients from trained DL models while maintaining the performance for other patients.

Privacy protection has become increasingly crucial in the field of epilepsy prediction. Some latest studies introduced the source free domain adaptation (SFDA), which only utilizes a pre-trained source model for protecting the source data privacy. However, the existing SFDA methods exist two shortcomings. (1) the offline setting, which is not suitable for real-world online scenarios (2) the poor performance, which is attributed to the absence of labeled calibration data during the adaptation phase. To this end, we proposed a online seizure prediction framework based on fine-tuning and test-time adaptation (FT3A). Specifically, FT3A employs one seizure event target data to fine-tune and continuously adapt pre-trained source model to unlabeled target data stream. In addition, the adaption and prediction is performed simultaneously. On the one hand, we design the task model as a multi-head structure to increase the confidence of the model and reduce error accumulation. On the other hand, a memory bank is introduced to store a small amount of historical EEG data, which helps handle the catastrophic forgetting concern of the model during online adaptation. Extensive experiments on public CHB-MIT dataset and the private freiburg hospital dataset indicate the superiority and generality of the proposed method.

Domain adaptation (DA) techniques are frequently utilized to enhance seizure prediction accuracy by leveraging the labeled electroencephalogram data of existing patients on new patients. Traditional DA methods, however, require access to the source domain while training the adaptation model, which poses a threat to sensitive patient information and privacy. To address this issue, in this article, we propose a novel Gaussian mixture modeling (GMM)-based source-free domain adaptation (GSFDA). Our method leverages the GMM joint source model and target data structure for clustering, employs uncertainty learning to minimize DA uncertainty, and uses the mixup technique to increase model robustness while reducing the impact of noisy pseudolabels. Notably, GSFDA only requires access to the source model parameters, and not the source domain, effectively safeguarding the privacy of patient information. This has substantial clinical implications for seizure prediction.

Arterial stiffness (AS) serves as a crucial indicator of arterial elasticity and function, typically requiring expensive equipment for detection. Given the strong correlation between AS and various photoplethysmography (PPG) features, PPG emerges as a convenient method for assessing AS. However, the limitations of independent PPG features hinder detection accuracy. This study introduces a feature selection method leveraging the interactive relationships between features to enhance the accuracy of predicting AS from a single-channel PPG signal. Initially, an adaptive signal interception method was employed to capture high-quality signal fragments from PPG sequences. 58 PPG features, deemed to have potential contributions to AS estimation, were extracted and analyzed. Subsequently, the interaction factor (IF) was introduced to redefine the interaction and redundancy between features. A feature selection algorithm (IFFS) based on the IF was then proposed, resulting in a combination of interactive features. Finally, the Xgboost model is utilized to estimate AS from the selected features set. The proposed approach is evaluated on datasets of 268 male and 124 female subjects, respectively. The results of AS estimation indicate that IFFS yields interacting features from numerous sources, rejects redundant ones, and enhances the association. The interaction features combined with the Xgboost model resulted in an MAE of 122.42 and 142.12 cm/sec, an SDE of 88.16 and 102.56 cm/sec, and a PCC of 0.88 and 0.85 for the male and female groups, respectively. The findings of this study suggest that the stated method improves the accuracy of predicting AS from single-channel PPG, which can be used as a non-invasive and cost-effective screening tool for atherosclerosis.

With the flourishing development of deep learning (DL) and the convolution neural network (CNN), electroencephalogram-based (EEG) emotion recognition is occupying an increasingly crucial part in the field of brain-computer interface (BCI). However, currently employed architectures have mostly been designed manually by human experts, which is a time-consuming and labor-intensive process. In this paper, we proposed a novel neural architecture search (NAS) framework based on reinforcement learning (RL) for EEG-based emotion recognition, which can automatically design network architectures. The proposed NAS mainly contains three parts: search strategy, search space, and evaluation strategy. During the search process, a recurrent network (RNN) controller is used to select the optimal network structure in the search space. We trained the controller with RL to maximize the expected reward of the generated models on a validation set and force parameter sharing among the models. We evaluated the performance of NAS on the DEAP and DREAMER dataset. On the DEAP dataset, the average accuracies reached 97.94%, 97.74%, and 97.82% on arousal, valence, and dominance respectively. On the DREAMER dataset, average accuracies reached 96.62%, 96.29% and 96.61% on arousal, valence, and dominance, respectively. The experimental results demonstrated that the proposed NAS outperforms the state-of-the-art CNN-based methods.

Emotion recognition based on electroencephalography (EEG) is a significant task in the brain-computer interface field. Recently, many deep learning-based emotion recognition methods are demonstrated to outperform traditional methods. However, it remains challenging to extract discriminative features for EEG emotion recognition, and most methods ignore useful information in channel and time. This article proposes an attention-based convolutional recurrent neural network (ACRNN) to extract more discriminative features from EEG signals and improve the accuracy of emotion recognition. First, the proposed ACRNN adopts a channel-wise attention mechanism to adaptively assign the weights of different channels, and a CNN is employed to extract the spatial information of encoded EEG signals. Then, to explore the temporal information of EEG signals, extended self-attention is integrated into an RNN to recode the importance based on intrinsic similarity in EEG signals. We conducted extensive experiments on the DEAP and DREAMER databases. The experimental results demonstrate that the proposed ACRNN outperforms state-of-the-art methods.

Recently, deep learning approaches have shown their advantages on solving scientific problems including full-wave nonlinear inverse problems. However, these data-driven methods face severe problems, especially about the low generalization ability, which means the trained models only work for scenarios with similar training data and physical setups. In this work, we propose a multi-domain weight-sharing method (MDWS) for inverse scattering problems, which increases the generalization ability of learning approaches for both data-based and physical-based out-of-range tests. Specifically, the proposed MDWS utilizes a physical layer of Green’s function to transform between induced current domain and electrical field domain, where weight-sharing blocks having the same weights in different incidences and stages are used to decouple the network structure from measurement setups. It is shown by intensive numerical and experimental tests both qualitatively and quantitatively that the proposed MDWS apparently outperforms the benchmarked method. Further, the proposed weight-sharing architecture also provides an efficient way to build large model in electromagnetic society with much less memory and computational cost.

In recent years, deep learning has gained widespread attention in electroencephalogram (EEG)-based emotion recognition. However, deep learning methods are usually time-consuming with a large amount of memory usage, which obstructs their practical usage on resource-constrained devices. In this paper, we propose a binary capsule network (Bi-CapsNet) for EEG emotion recognition with low computational cost and memory usage. The Bi-CapsNet binarizes 32-bit weights and activations to 1 b, and replaces floating-point operations with efficient bitwise operations. To address the issue of function discontinuity in backward propagation, we use a continuous function to approximate the binarization process. Two popular EEG emotion databases, namely, DEAP and DREAMER, are used for performance evaluation. In comparison to its full-precision counterpart, the Bi-CapsNet achieves a $>\!25\times$reduction on the computational cost and a $>\!5\times$ reduction on the memory usage, while with only a $< $1% drop on the recognition accuracy. Compared to some state-of-the-art EEG emotion recognition methods, the proposed method obtains more competitive performance. In addition, the Bi-CapsNet is implemented on a mobile phone via an open-source binary inference framework named Bolt, and it achieves an $\sim\! 5\times$ inference acceleration in comparison to its full-precision counterpart.

Video-based Photoplethysmography (VPPG) can identify arrhythmic pulses during atrial fibrillation (AF) from facial videos, providing a convenient and cost-effective way to screen for occult AF. However, facial motions in videos always distort VPPG pulse signals and thus lead to the false detection of AF. Photoplethysmography (PPG) pulse signals offer a possible solution to this problem due to the high quality and resemblance to VPPG pulse signals. Given this, a pulse feature disentanglement network (PFDNet) is proposed to discover the common features of VPPG and PPG pulse signals for AF detection. Taking a VPPG pulse signal and a synchronous PPG pulse signal as inputs, PFDNet is pre-trained to extract the motion-robust features that the two signals share. The pre-trained feature extractor of the VPPG pulse signal is then connected to an AF classifier, forming a VPPG-driven AF detector after joint fine-tuning. PFDNet has been tested on 1440 facial videos of 240 subjects (50% AF absence and 50% AF presence). It achieves a Cohen's Kappa value of 0.875 (95% confidence interval: 0.840-0.910, P<0.001) on the video samples with typical facial motions, which is 6.8% higher than that of the state-of-the-art method. PFDNet shows significant robustness to motion interference in the video-based AF detection task, promoting the development of opportunistic screening for AF in the community.

Sparse unmixing (SU) of hyperspectral image (HSI), as a semisupervised approach, aims to find the optimal subset of the spectral library known in advance to represent each pixel in HSI. However, most of the existing SU methods cannot take full advantage of spatial information and mixed noise in HSI. To this end, we propose a superpixel-based noise-robust SU method (SNRSU) in the presence of mixed noise. First, we perform superpixel segmentation (SS) on the first principal component of HSI to extract the homogeneous regions. Then, we unmix each superpixel based on sparse representation (SR) and low-rank representation (LRR) in the maximuma posterioriframework, which can make full use of the spatial–spectral information in HSI under complex mixed noise. A number of experiments on simulated and real HSI datasets confirm the superior performance of the proposed SNRSU both qualitatively and quantitatively.

Spatially uneven illuminations are the dominant interference of video-based heart rate (HR) screening for cooperated subjects in a telehealth service. In this letter, a remote photoplethysmography (rPPG) method is introduced to stably extract pulsatile signals against uneven facial illuminations based on the multivariate singular spectrum analysis (MSSA). This method first divides the facial skins into multiple patches, where the hue channels resistant to light intensity variations are prepared from selected optimal patches. Considering the spatial correlations of heartbeats, the hue signals are then decomposed using the MSSA to reconstruct pulses. Finally, the HR is determined as the one with the highest ratio of energy around the dominant frequency from the first group of MSSA reconstructed signals. Experimental results demonstrate the effectiveness of the proposed method on the in-house BSIPL-rPPG database and the public COHFACE database, where the correlation coefficients of the estimated HRs achieve 0.95 and 0.98, respectively, outperforming those of the comparison methods.

In this paper, an unrolling algorithm of the iterative subspace-based optimization method (SOM) is proposed for solving full-wave inverse scattering problems (ISPs). The unrolling network, named SOM-Net, inherently embeds the Lippmann-Schwinger physical model into the design of network structures. The SOM-Net takes the deterministic induced current and the raw permittivity image obtained from back-propagation (BP) as the input. It then updates the induced current and the permittivity successively in sub-network blocks of the SOM-Net by imitating iterations of the SOM. The final output of the SOM-Net is the full predicted induced current, from which the scattered field and the permittivity image can also be deduced analytically. The parameters of the SOM-Net are optimized in a supervised manner with the total loss to simultaneously ensure the consistency of the induced current, the scattered field, and the permittivity in the governing equations. Numerical tests on both synthetic and experimental data verify the superior performance of the proposed SOM-Net over typical ones. The results on challenging examples like scatterers with tough profiles or high permittivity demonstrate the good generalization ability of the SOM-Net. With the use of deep unrolling technology, this work builds a bridge between traditional iterative methods and deep learning methods for solving ISPs.

The end-to-end scalable cascaded convolutional neural networks (SC-CNNs) are proposed to solve inverse scattering problems (ISPs), and the high-resolution image can be directly obtained from the scattered field with the guiding by multiresolution labels in the cascaded blocks. To alleviate the difficulty of solving the ISPs via a full-wave way, the proposed SC-CNNs are physically decomposed into two parts, i.e., the linear transformation and the multiresolution imaging networks. The first part is composed of one CNN block and is used to mimic the linear transformation [e.g., backpropagation (BP)] from scattered field to the preliminary image, whereas the second part consists of a few cascaded CNN blocks to realize the reconstruction from the rough image to high-resolution image. With more high-frequency components incorporating into the multiresolution labels, the cascaded networks can be guided through those labels, avoiding black-box operations and enhancing the physical meaning and interpretability. The proposed SC-CNNs are verified by both the synthetic and experimental examples and it is proved that better performance can be achieved in terms of both inversion accuracy and efficiency compared to the BP-Unet and direct inversion scheme (DIS).

Recently, deep neural networks (DNNs) have been applied to emotion recognition tasks based on electroencephalography (EEG), and have achieved better performance than traditional algorithms. However, DNNs still have the disadvantages of too many hyperparameters and lots of training data. To overcome these shortcomings, in this article, we propose a method for multi-channel EEG-based emotion recognition using deep forest. First, we consider the effect of baseline signal to preprocess the raw artifact-eliminated EEG signal with baseline removal. Secondly, we construct 2 D frame sequences by taking the spatial position relationship across channels into account. Finally, 2 D frame sequences are input into the classification model constructed by deep forest that can mine the spatial and temporal information of EEG signals to classify EEG emotions. The proposed method can eliminate the need for feature extraction in traditional methods and the classification model is insensitive to hyperparameter settings, which greatly reduce the complexity of emotion recognition. To verify the feasibility of the proposed model, experiments were conducted on two public DEAP and DREAMER databases. On the DEAP database, the average accuracies reach to 97.69% and 97.53% for valence and arousal, respectively; on the DREAMER database, the average accuracies reach to 89.03%, 90.41%, and 89.89% for valence, arousal and dominance, respectively. These results show that the proposed method exhibits higher accuracy than the state-of-art methods.

Remote photoplethysmography (rPPG) is a non-contact technique for measuring cardiac signals from facial videos. High-quality rPPG pulse signals are urgently demanded in many fields, such as health monitoring and emotion recognition. However, most of the existing rPPG methods can only be used to get average heart rate (HR) values due to the limitation of inaccurate pulse signals. In this paper, a new framework based on generative adversarial network, called PulseGAN, is introduced to generate realistic rPPG pulse signals through denoising the chrominance (CHROM) signals. Considering that the cardiac signal is quasi-periodic and has apparent time-frequency characteristics, the error losses defined in time and spectrum domains are both employed with the adversarial loss to enforce the model generating accurate pulse waveforms as its reference. The proposed framework is tested on three public databases. The results show that the PulseGAN framework can effectively improve the waveform quality, thereby enhancing the accuracy of HR, the interbeat interval (IBI) and the related heart rate variability (HRV) features. The proposed method significantly improves the quality of waveforms compared to the input CHROM signals, with the mean absolute error of AVNN (the average of all normal-to-normal intervals) reduced by 41.19%, 40.45%, 41.63%, and the mean absolute error of SDNN (the standard deviation of all NN intervals) reduced by 37.53%, 44.29%, 58.41%, in the cross-database test on the UBFC-RPPG, PURE, and MAHNOB-HCI databases, respectively. This framework can be easily integrated with other existing rPPG methods to further improve the quality of waveforms, thereby obtaining more reliable IBI features and extending the application scope of rPPG techniques.

In this paper, we combine two techniques together, i.e., the fast Fourier transform-twofold subspace-based optimization method (FFT-TSOM) and multiplicative regularization (MR) to solve inverse scattering problems. When applying MR to the objective function in the FFT-TSOM, the new method is referred to as MR-FFT-TSOM. In MR-FFT-TSOM, a new stable and effective strategy of regularization has been proposed. MR-FFT-TSOM inherits not only the advantages of the FFT-TSOM, i.e., lower computational complexity than the TSOM, better stability of the inversion procedure, and better robustness against noise compared with the SOM, but also the edge-preserving ability from the MR. In addition, a more relaxed condition of choosing the number of current bases being used in the optimization can be obtained compared with the FFT-TSOM. Particularly, MR-FFT-TSOM has even better robustness against noise compared with the FFT-TSOM and multiplicative regularized contrast source inversion (MR-CSI). Numerical simulations including both inversion of synthetic data and experimental data from the Fresnel data set validate the efficacy of the proposed algorithm.

Subspace techniques have been introduced in the framework of contrast source (CS) extended born (CSEB) model, for improving its reconstruction capabilities. Two techniques are demonstrated. First, a scheme for generating a good initial guess of the scatterer profile is shown. Second, subspace-based optimization method is used for optimization. Using the suggested techniques, CSEB model can be applied for solving inverse electromagnetic scattering problem with an extended range of application with respect to previous contributions, particularly for very high contrast lossy scatterers.