Previous studies have shown that emotions elicit responses in various physiological systems that underlie emotional experience. However, these studies have focused almost solely on how emotions alter the activity of individual physiological systems. Moreover, recent research has highlighted that the human organism is an integrated entity, wherein physiological systems constantly interact to form a dynamic network. Therefore, in this study, we attempted to evaluate whether the physiological network changes with different emotions. We employed multivariate transfer entropy to assess the interaction between six types of emotion-related physiological signals and subsequently constructed physiological networks across forty emotional trials. Additionally, topological parameters were applied to each physiological network to quantify its overall structure. The results revealed that emotion has a significant impact on the topology of physiological networks, indicating the integrated behavior of physiological systems in response to emotion at the macro-scale level. Furthermore, different physiological systems may play diverse roles within the physiological network. In summary, we propose a system-level integrative approach, referred to as the physiological network, to study the physiological responses to emotion. This not only contributes to a deeper understanding of emotional mechanisms but also demonstrates potential for disease research and engineering applications.

This study introduces PicoSleepNet, an ultra-lightweight sleep stage classification method that utilizes a spiking neural network (SNN) with single-channel electroencephalogram (EEG) signals. Traditional methods use multi-bit Nyquist sampling and dense computing, which result in high complexity and power consumption, hindering their deployment on wearable devices. To address these limitations, we propose an innovative pipeline combining single-bit sub-Nyquist level-crossing sampling (LCS) and sparse computing based on SNN. First, LCS adaptively encodes EEG signals into event-driven spike sequences, reducing data volume by 6.98× while preserving essential signal characteristics compared to Nyquist sampling. Second, a sparse recurrent spiking neural network (RSNN) architecture, optimized by the masked backpropagation and sparse regularization (Masked-BPSR) technique, improves performance and reduces computational costs. Third, quantization-aware training (QAT) ensures that the model maintains high accuracy with low-bitwidth quantization, significantly reducing computational power consumption and enabling hardware-friendly deployment. Compared with current state-of-the-art sleep staging approaches, PicoSleepNet achieves competitive performance on three public datasets (Sleep-EDF-20, Sleep-EDF-78, and ISRUC-Sleep) with accuracies of 83.5%, 77.9%, 79.4% and macro-F1 scores of 75.2%, 68.1%, 77.2%, respectively. Meanwhile, by leveraging the computational sparsity design of RSNN and the joint optimization of Masked-BPSR and QAT, PicoSleepNet achieves an ultra-lightweight model with only 14.0-25.8 K parameters (reduced by nearly 2×) and 681.4-842.0 K operations (reduced by 27×), reducing computational power consumption by 1480×. This approach demonstrates the feasibility of deploying ultra-lightweight sleep staging systems in wearable devices and neuromorphic hardware, paving the way for broader applications in real-time health monitoring.

Transformer models cannot easily scale to long sequences due to their O(N 2 ) time and space complexity.This has led to Transformer variants seeking to lower computational complexity, such as Longformer and Performer.While such models have theoretically greater efficiency, their effectiveness on real NLP tasks has not been well studied.We benchmark 7 variants of Transformer models on 5 difficult NLP tasks and 7 datasets.We design experiments to isolate the effect of pretraining and hyperparameter settings, to focus on their capacity for long-range attention.Moreover, we present various methods to investigate attention behaviors to illuminate model details beyond metric scores.We find that the modified attention in long-range transformers has advantages on content selection and query-guided decoding, but they come with previously unrecognized drawbacks such as insufficient attention to distant tokens and accumulated approximation error.

Recent studies have shown that using an external Language Model (LM) benefits the end-to-end Automatic Speech Recognition (ASR).However, predicting tokens that appear less frequently in the training set is still quite challenging.The longtail prediction problems have been widely studied in many applications, but only been addressed by a few studies for ASR and LMs.In this paper, we propose a new memory augmented lookup dictionary based Transformer architecture for LM.The newly introduced lookup dictionary incorporates rich contextual information in training set, which is vital to correctly predict long-tail tokens.With intensive experiments on Chinese and English data sets, our proposed method is proved to outperform the baseline Transformer LM by a great margin on both word/character error rate and tail tokens error rate.This is achieved without impact on the decoding efficiency.Overall, we demonstrate the effectiveness of our proposed method in boosting the ASR decoding performance, especially for longtail tokens.

Multilingual pretrained models have shown strong cross-lingual transfer ability.Some works used code-switching sentences, which consist of tokens from multiple languages, to enhance the cross-lingual representation further, and have shown success in many zero-shot cross-lingual tasks.However, code-switched tokens are likely to cause grammatical incoherence in newly substituted sentences, and negatively affect the performance on tokensensitive tasks, such as Part-of-Speech (POS) tagging and Named-Entity-Recognition (NER).This paper mitigates the limitation of the codeswitching method by not only making the token replacement but considering the similarity between the context and the switched tokens so that the newly substituted sentences are grammatically consistent during both training and inference.We conduct experiments on crosslingual POS and NER over 30+ languages, and demonstrate the effectiveness of our method by outperforming the mBERT by 0.95 and original code-switching method by 1.67 on F1 scores.How do menschen look at and अनु भव 艺术 ?How do people look at and experience art ?How do people look at and experience art ?How do Menschen look at and अनु भव 艺术 ?(b): Ours (a): Code-Switched Training Sentence SCONJ AUX NOUN VERB ADP CCPNJ VERB NOUN PUNCT

Building pretrained language models is considered expensive and data-intensive, but must we increase dataset size to achieve better performance?We propose an alternative to larger training sets by automatically identifying smaller yet domain-representative subsets.We extend Cynical Data Selection, a statistical sentence scoring method that conditions on a representative target domain corpus.As an example, we treat the OntoNotes corpus as a target domain and pretrain a RoBERTa-like encoder from a cynically selected subset of the Pile.On both perplexity and across several downstream tasks in the target domain, it consistently outperforms random selection with 20x less data, 3x fewer training iterations, and 2x less estimated cloud compute cost, validating the recipe of automatic document selection for LM pretraining.

OBJECTIVE: Based on cybernetics, a large system can be divided into subsystems, and the stability of each can determine the overall properties of the system. However, this stability analysis perspective has not yet been employed in electrocardiogram (ECG) signals. This is the first study to attempt to evaluate whether the stability of decomposed ECG subsystems can be analyzed in order to effectively investigate the overall performance of ECG signals, and aid in disease diagnosis. METHODS: We used seven different cardiac pathologies (myocardial infarction, cardiomyopathy, bundle branch block, dysrhythmia, hypertrophy, myocarditis, and valvular heart disease) to illustrate our method. Dynamic mode decomposition (DMD) was first used to decompose ECG signals into dynamic modes (DMs) which can be regarded as ECG subsystems. Then, the features related to the DMs stabilities were extracted, and nine common classifiers were implemented for classification of these pathologies. RESULTS: Most features were significant for differentiating the above-mentioned groups (p value<0.05 after Bonferroni correction). In addition, our method outperformed all existing methods for cardiac pathology classification. CONCLUSION: We have provided a new spatial and temporal decomposition method, namely DMD, to study ECG signals. SIGNIFICANCE: Our method can reveal new cardiac mechanisms, which can contribute to the comprehensive understanding of its underlying mechanisms and disease diagnosis, and thus, can be widely used for ECG signal analysis in the future.

Chenlong Hu, Yukun Feng, Hidetaka Kamigaito, Hiroya Takamura, Manabu Okumura. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. 2021.

OBJECTIVE: Previous studies have already shown that electroencephalography (EEG) brain network (BN) can reflect the health status of individuals. However, novel methods are still needed for BN analysis. Therefore, in this study, BNs were constructed based on stable and unstable EEG components, and these may be implemented for disease diagnosis. METHODS: Parkinson's disease (PD) was used as an example to illustrate this method. First, EEG signals were decomposed into dynamic modes (DMs). Each DM contains one eigenvalue that can determine not only the stability of that mode, but also its corresponding oscillatory frequency. Second, the stable and unstable components of EEG signals in each frequency band (delta, theta, alpha and beta) were calculated, which are based on the stable and unstable DMs within each respective frequency band. Third, newly developed BNs were constructed, including stable brain network (SBN), unstable brain network (UBN) and inter-connected brain network (IBN). Finally, their topological attributes were extracted in order to differentiate between PD patients and healthy controls (HC). Furthermore, topological attributes were also derived from traditional brain network (TBN) for comparison. RESULTS: Most topological attributes of SBN, UBN and IBN can significantly differentiate between PD patients and HC ( p value 0.05). Furthermore, the area under the curve (AUC), precision and recall values of SBN analysis are all significantly higher than TBN. CONCLUSION: We proposed a new perspective on EEG BN analysis. SIGNIFICANCE: These newly developed BNs not only have biological significance, but also could be widely applied in most medical and engineering fields.

We address the task of generating live soccer-match commentaries from play event data. This task has characteristics that (i) each commentary is only partially aligned with events, (ii) play event data contains many types of categorical and numerical attributes, (iii) live commentaries often mention player names and team names. For these reasons, we propose an encoder for play event data, which is enhanced with a gate mechanism. We also introduce an attention mechanism on events. In addition, we introduced placeholders and their reconstruction mechanism to enable the model to copy appropriate player names and team names from the input data. We conduct experiments on the play data of the English Premier League, provide a discussion on the result including generated commentaries.

We propose a simple and effective method to inject word-level information into characteraware neural language models.Unlike previous approaches which usually inject wordlevel information at the input of a long shortterm memory (LSTM) network, we inject it into the softmax function.The resultant model can be seen as a combination of characteraware language model and simple word-level language model.Our injection method can also be used together with previous methods.Through the experiments on 14 typologically diverse languages, we empirically show that our injection method, when used together with the previous methods, works better than the previous methods, including a gating mechanism, averaging, and concatenation of word vectors.We also provide a comprehensive comparison of these injection methods.

The nature of spike coding of auditory signals is studied by comparing mean firing rate codes with conventional approaches in speech recognition tasks. Mean firing rate spike representations are problematic since most auditory nerve fibers are saturated at typical conversation levels. However, these problems are eased somewhat when it is considered that there are other nerve fibers (e.g. low spontaneous firing rate fibers) that could efficiently encode the information at each channel of the cochlea. We show that window-based, mean firing rate features can be used with a crude cochlea model to achieve the same level of performance as a conventional MFCC-based approach. These results assume that there are sufficient neurons available at each channel to average the randomness of the stochastic spike trains. Furthermore, we argue that the mean firing rate features could be augmented with timing cues for further performance improvement.