While end-to-end multi-channel electroencephalography (EEG) learning approaches have shown significant promise, their applicability is often constrained in neurological diagnostics, such as intracranial EEG resources. When provided with a single-channel EEG, how can we learn representations that are robust to multi-channels and scalable across varied tasks, such as seizure prediction? In this paper, we present SplitSEE, a structurally splittable framework designed for effective temporal-frequency representation learning in single-channel EEG. The key concept of SplitSEE is a self-supervised framework incorporating a deep clustering task. Given an EEG, we argue that the time and frequency domains are two distinct perspectives, and hence, learned representations should share the same cluster assignment. To this end, we first propose two domain-specific modules that independently learn domain-specific representation and address the temporal-frequency tradeoff issue in conventional spectrogram-based methods. Then, we introduce a novel clustering loss to measure the information similarity. This encourages representations from both domains to coherently describe the same input by assigning them a consistent cluster. SplitSEE leverages a pretraining-to-fine-tuning framework within a splittable architecture and has following properties: (a) Effectiveness: it learns representations solely from single-channel EEG but has even outperformed multi-channel baselines. (b) Robustness: it shows the capacity to adapt across different channels with low performance variance. Superior performance is also achieved with our collected clinical dataset. (c) Scalability: With just one fine-tuning epoch, SplitSEE achieves high and stable performance using partial model layers.

The Transformer model has shown leading performance in time series forecasting. Nevertheless, in some complex scenarios, it tends to learn low-frequency features in the data and overlook high-frequency features, showing a frequency bias. This bias prevents the model from accurately capturing important high-frequency data features. In this paper, we undertake empirical analyses to understand this bias and discover that frequency bias results from the model disproportionately focusing on frequency features with higher energy. Based on our analysis, we formulate this bias and propose Fredformer, a Transformer-based framework designed to mitigate frequency bias by learning features equally across different frequency bands. This approach prevents the model from overlooking lower amplitude features important for accurate forecasting. Extensive experiments show the effectiveness of our proposed approach, which can outperform other baselines in different real-world time-series datasets. Furthermore, we introduce a lightweight variant of the Fredformer with an attention matrix approximation, which achieves comparable performance but with much fewer parameters and lower computation costs. The code is available at: https://github.com/chenzRG/Fredformer

Electroencephalography (EEG) is a major tool for studying neurophysiological processes. Investigating reliable representations from highly noisy measurements is a pending challenge, however, the medically treasured and insufficient labeled data have driven this process away from a supervised learning manner. Recent works have turned their attention to self-supervised learning (SSL), putting the contrastive strategy on capturing the spatio-temporal characteristics of the neuronal events of interest. We argue that the temporal-spatial view is not the best choice for the SSL contrastive objective because there is a missing piece of the EEG representation that is usually ignored: dynamic fluctuations in brain neurons and the statistical learning of analog/artificial neural networks cannot handle the dynamic characteristics well. This paper proposes a novel two-view contrastive learning framework to refine EEG features from local-global and past-future views. An array of spiking neural networks is embedded to project spatio-temporal features onto the spike sequences to represent the dynamic fluctuation information of EEG. Experimenting with sleep stage classification and prediction of lethal epileptic seizures, we verify the proposal competes favorably against the state-of-the-art methods and offers high-quality features, that is, supervised learning on top of them observes a significant improvement in classification after only one training iteration.