Users’ interactions with recommender systems often involve more than simple acceptance or rejection. We highlight two overlooked states: hesitation, when people deliberate without certainty, and tolerance, when this hesitation escalates into unwanted engagement before ending in disinterest. Across two large-scale surveys (N = 6, 644 and N = 3, 864), hesitation was nearly universal, and tolerance emerged as a recurring source of wasted time, frustration, and diminished trust. Analyses of e-commerce and short-video platforms confirm that tolerance behaviors, such as clicking without purchase or shallow viewing, correlate with decreased activity. Finally, an online field study at scale shows that even lightweight strategies treating tolerance as distinct from interest can improve retention while reducing wasted effort. By surfacing hesitation and tolerance as consequential states, this work reframes how recommender systems should interpret feedback, moving beyond clicks and dwell time toward designs that respect user value, reduce hidden costs, and sustain engagement.

Explaining deep models for time-series data is crucial for identifying key patterns in sensitive domains, such as healthcare and finance. However, due to the lack of unified optimization criterion, existing explanation methods often suffer from redundancy and incompleteness, where irrelevant patterns are included or key patterns are missed in explanations. To address this challenge, we propose the Optimal Information Retention Principle, where conditional mutual information defines minimizing redundancy and maximizing completeness as optimization objectives. We then derive the corresponding objective function theoretically. As a practical framework, we introduce an explanation framework ORTE, learning a binary mask to eliminate redundant information while mining temporal patterns of explanations. We decouple the discrete mapping process to ensure the stability of gradient propagation, while employing contrastive learning to achieve precise filtering of explanatory patterns through the mask, thereby realizing a trade-off between low redundancy and high completeness. Extensive quantitative and qualitative experiments on synthetic and real-world datasets demonstrate that the proposed principle significantly improves the accuracy and completeness of explanations compared to baseline methods. The code is available at https://github.com/moon2yue/ORTE_public.

Incomplete time series classification is both practically valuable and challenging as missing values in time series data are prevalent in real-world scenarios. Current approaches suffer from two major limitations. First, they overemphasize the consistency of data reconstruction during missing value imputation while neglecting the task-effectiveness of the imputed results for the classification. Second, they fail to systematically establish a synergistic optimization mechanism between data imputation and feature representation. To address these challenges, we propose a Hierarchical Conditional Information Bottleneck (HCIB) framework, which achieves incomplete time series classification through end-to-end joint optimization. Specifically, at the data imputation level, we re-examine the dual effects of missing data: the loss of critical information (Loss) versus the gain in interference suppression (Gain), elucidating this duality through bias-variance trade-off theory. Building on this analysis, we propose a task-information sufficiency criterion and extend the information bottleneck theory into a task-driven imputation framework by incorporating label information as a conditional constraint. At the feature representation level, we construct a hierarchical information bottleneck architecture to learn compressed yet informative temporal representations from the task-oriented imputed data. Furthermore, we derive the optimizable objective function for HCIB and design specialized neural network architectures for time series. Comprehensive experiments on multivariate and univariate time series datasets across multiple domains consistently demonstrate that the proposed method achieves significant improvements in classification performance compared to SOTA approaches.

Emotion recognition based on multimodal physiological signals is playing an increasingly important role in areas such as human-computer interaction and disease diagnosis, attracting growing attention from the research community. Current studies primarily focus on emotion recognition under unified data collection paradigms, overlooking the prevalent issue of imperfect modality matching in real-world scenarios. In particular, existing methods fail to effectively utilize these mismatched modalities, leading to incomplete emotional representations. This limits the model's ability to accurately capture the multidimensional semantic features of emotions, thereby constraining its effectiveness and applicability in practical settings. To address this challenge, we propose MoCERNet. At the modality level, it first reduces the domain gap among matched modalities and then aligns mismatched modalities in a semantics-aware manner, guided by the matched ones. At the decision level, it further mitigates the global distribution discrepancies to achieve a more complete emotional representation. In addition, we design a Nervous System Functional Structure Transformer (NFSformer) that enables the model to focus on the correlation between different brain regions and peripheral physiological signals under various emotional states, thereby enhancing its capacity to model complex emotional processes. Experiments on three multimodal emotion datasets demonstrate that MoCERNet outperforms state-of-the-art baselines under imperfect modality matching scenarios.

Medical time-series analysis differs fundamentally from general ones by requiring specialized domain knowledge to interpret complex signals and clinical context. Large language models (LLMs) hold great promise for augmenting medical time-series analysis by complementing raw series with rich contextual knowledge drawn from biomedical literature and clinical guidelines. However, realizing this potential depends on precise and meaningful prompts that guide the LLM to key information. Yet, determining what constitutes effective prompt content remains non-trivial—especially in medical settings where signal interpretation often hinges on subtle, expert-defined decision-making indicators. To this end, we propose InDiGO, a knowledge-aware evolutionary learning framework that integrates clinical signals and decision-making indicators through iterative optimization. Across four medical benchmarks, InDiGO consistently outperforms prior methods. The code is available at: https://github.com/jinxyBJTU/InDiGO.

Affective brain-computer interfaces (aBCIs) play a crucial role in personalized human–computer interaction and neurofeedback modulation. To develop practical and effective aBCI paradigms and to investigate the spatial-temporal dynamics of brain activity under emotional inducement, portable electroencephalography (EEG) signals have been widely adopted. To further enhance spatial-temporal perception, functional near-infrared spectroscopy (fNIRS) has attracted increasing interest in the aBCI field and has been explored in combination with EEG. However, existing datasets typically provide only static fixation labels, overlooking the dynamic changes in subjects' emotions. Notably, some studies have attempted to collect continuously annotated emotional data, but they have recorded only peripheral physiological signals without directly observing brain activity, limiting insight into underlying neural states under different emotions. To address these challenges, we present the Real-time labeled EEG-fNIRS Dataset (REFED). To the best of our knowledge, this is the first EEG-fNIRS dataset with real-time dynamic emotional annotations. REFED simultaneously records brain signals from both EEG and fNIRS modalities while providing continuous, real-time annotations of valence and arousal. The results of the data analysis demonstrate the effectiveness of emotion inducement and the reliability of real-time annotation. This dataset offers the possibility for studying the neurovascular coupling mechanism under emotional evolution and for developing dynamic, robust affective BCIs.

The study of the relationship between emotions and physiological signals of subjects under multimedia stimulation is an emerging field, and many important advances are made. However, there are still some challenges: 1) How to effectively utilize the complementarity among spatial-spectral-temporal domain information. 2) How to employ the heterogeneity and the correlation among multi-modal physiological signals simultaneously. 3) How to improve the robustness of the model dealing with missing channels. 4) How to model the dependency among different emotions. In this paper, we propose a novel two-stream Dynamic Heterogeneous Graph Recurrent Neural Network called DHGRNN. Specifically, DHGRNN consists of a spatial-temporal stream, a spatial-spectral stream, a fusion layer, and a multi-label classifier. Each stream is composed of a graph transformer network, evolved graph convolutional neural network, and gated recurrent units. We propose a graph-based two-stream structure to fuse the information of the spatial-spectral-temporal domain simultaneously. Graph transformer network and evolved graph convolutional neural network are used to model the heterogeneity and correlation of multi-modal physiological signals, respectively. To deal with the problem of robustness in the face of missing channel data, we transform it into the problem of dynamic graphs and use a dynamic graph neural network to improve the robustness. In addition, we propose a multi-label classifier to model the dependency among different emotion dimensions. Experiments on three public datasets demonstrate that our proposed model outperforms existing state-of-the-art methods.

Sleep stage classification is an important step in the diagnosis and treatment of sleep disorders. Despite the high classification performance of previous sleep stage classification work, some challenges remain unresolved: 1) How to effectively capture salient waves in sleep signals to improve sleep stage classification results. 2) How to capture salient waves affected by inter-subject variability. 3) How to adaptively regulate the importance of different modals for different sleep stages. To address these challenges, we propose SleepWaveNet, a multimodal salient wave detection network, which is motivated by the salient object detection task in computer vision. It has a U-Transformer structure to detect salient waves in sleep signals. Meanwhile, the subject-adaptation wave extraction architecture based on transfer learning can adapt to the information of target individuals and extract salient waves with inter-subject variability. In addition, the multimodal attention module can adaptively enhance the importance of specific modal data for sleep stage classification tasks. Experiments on three datasets show that SleepWaveNet has better overall performance than existing baselines. Moreover, visualization experiments show that the model has the ability to capture salient waves with inter-subject variability.

Self-supervised contrastive learning (SCL) has driven significant advancements in time-series representation learning. While recent studies built upon the information noise contrastive estimation (InfoNCE) loss framework focus on constructing appropriate positives and negatives, we theoretically analyze and identify two overlooked issues inherent in this approach: false negatives and class imbalance. To address these challenges, we propose a simple yet effective modification based on the SimCLR framework, integrating a multi-instance discrimination task to mitigate false negatives. Additionally, we introduce a graph-based interactive projection head and semantic consistency regularization, which enhances minority-class representations with minimal annotation cost. Extensive experiments on six real-world time-series datasets demonstrate that our approach consistently outperforms state-of-the-art methods, achieving up to 3.96% higher accuracy and 10.73% improvement in $F1$ -score, particularly benefiting imbalanced data scenarios.

Given a query, the task of Natural Language Video Localization (NLVL) is to localize a temporal moment in an untrimmed video that semantically matches the query.In this paper, we adopt a proposal-based solution that generates proposals (i.e., candidate moments) and then select the best matching proposal.On top of modeling the cross-modal interaction between candidate moments and the query, our proposed Moment Sampling DETR (MS-DETR) enables efficient moment-moment relation modeling.The core idea is to sample a subset of moments guided by the learnable templates with an adopted DETR (DEtection TRansformer) framework.To achieve this, we design a multiscale visual-linguistic encoder, and an anchorguided moment decoder paired with a set of learnable templates.Experimental results on three public datasets demonstrate the superior performance of MS-DETR. 1

Sleep stage classification based on physiological time-series is essential for sleep quality evaluation and the diagnosis of sleep disorders in clinical practice. Existing machine learning studies have achieved adequate results in sleep stage classification. However, those methods neglect the significance of simultaneously capturing the interactivity and heterogeneity of physiological signals. In this paper, we propose a novel Sleep Heterogeneous Graph Neural Network (SleepHGNN) to employ these essential features. The SleepHGNN is a deep graph network consisting of Heterogeneous Graph Transformer layers, which are composed of a Heterogeneous Message Passing module for capturing the heterogeneity and a Target-Specific Aggregation module for capturing the interactivity of physiological signals. The experiments show that the SleepHGNN outperforms the state-of-the-art models on the sleep stage classification task. The source code of SleepHGNN is available at: https://github.com/zhouyh310/SleepHGNN.

With the deepening of neuroscience research, data mining of brain signals is becoming an emerging topic. Among various brain signals, electroencephalography (EEG) has attracted more and more attention due to its advantages of non-invasiveness, portability, and low cost. EEG modeling and analysis play a vital role in human healthcare. Although many machine learning algorithms have been successfully applied to data mining of EEG signals, few of them achieve a win-win in classification performance and interpretability. In this paper, we propose a Bayesian graph neural network named BayesEEGNet. Considering an electrical impulse between two nodes in the brain as a Poisson process, the countless electrical impulses generated by the brain in a period are represented as an infinite number of connection probability graphs. After coupling and transforming these probability graphs, we interpret the brain’s electrical activity state as the brain’s perceptual state. Benefiting from the joint optimization of Bayesian modules and deep neural networks, our model shows superior classification performance in sleep stage classification and emotion recognition tasks. Meanwhile, our model is able to learn interpretable functional connectivity relationships between EEG channels without any prior knowledge.

Contrastive learning has successfully addressed the scarcity of large-scale labeled datasets, especially in the physiological time series field. Existing methods construct easy positive pairs as substitutes for ground truth based on temporal dynamics or instance consistency. Despite the potential of hard positive samples to provide richer gradient information and facilitate the acquisition of more discriminative representations, they are frequently overlooked in sampling strategies, thus constraining the classification capacity of models. In this paper, we focus on multi-view physiological signals and propose a novel hard positive sampling strategy based on the view consistency. Multi-view signals are recorded from sensors attached to different organs of human body. Additionally, we propose a Multi-View Consistency Contrastive (MVCC) learning framework to jointly extract intra-view temporal dynamics and inter-view consistency features. Experiments have been carried out on two public datasets and our method demonstrates state-of-the-art performance, achieving 83.25% and 73.37% accuracy on SleepEDF and ISRUC, respectively.

Electroencephalogram (EEG) signals for motor imagery (MI) are easily influenced by the environment and the state of the subject, which exhibit temporal and spatial variance. And this variance is more significant across subjects and sessions, which imposes limitations on the cross-domain MI tasks. To address this problem, we propose a Multi-level Spatial-Temporal Adaptation Network (MSTAN), extracting domain-invariant multi-level spatial-temporal features to overcome domain differences. First, stacked spatial-temporal graph convolution (STGCN) layers and an attention-based readout module are designed to extract spatial-temporal patterns of EEGs at multiple levels. An adaptation scheme is then introduced to narrow domain differences: 1) Individual graph parameters for the source and target domains are designed at each STGCN layer to capture the domain-specific brain region dynamic relationships; 2) The differences of spatial-temporal features between the source and target domain are reduced by minimizing the distribution distance. Experiments are conducted to evaluate the proposed method on a public dataset and the results show that our method achieves state-of-the-art performance in cross-domain motor imagery classification.

Although supervised deep learning methods achieve favorable performance in automatic sleep staging, they are limited in clinical situations due to the heavy reliance on massive labeled multi-channel polysomnogram (PSG) recordings. Accordingly, to alleviate the reliance on labeled PSG, we present SleepECL, a self-supervised learning framework based on electroencephalogram (EEG) signals taking advantage of contrastive learning, which learns efficient representations by contrasting semantically consistent and inconsistent samples (a.k.a. positive samples and negative samples). Specifically, SleepECL conducts contrastive learning upon local representations (i.e., intra-epoch EEG decoding) as well as contextual representations (i.e., interepoch dependency) and incorporates sleep expert knowledge to discover more accurate positive samples in contrastive learning, leading to more effective representations. Experimental results on two publicly available datasets demonstrate that our SleepECL outperforms state-of-the-art self-supervised methods. Moreover, the pre-trained model achieves acceptable performance using only a few label single-channel EEG recordings, which contributes to a more convenient application of automatic sleep staging in clinical situations.

Sleep stage classification is critical for diagnosing sleep quality. While deep neural networks are becoming popular for automatic sleep stage classification with supervised learning, large-scale labeled datasets are still hard to acquire. Recently, self-supervised learning (SSL) has become one of the most prominent approaches to alleviate the burden of labeling works. However, existing SSL methods are mainly designed for non-temporally correlated data and only learning representations from an instance level. Hence, the objective of this paper is to learn robust and generalizable representations for physiological signals with self-supervised learning. Specifically, we make the following contributions: (1) we propose a novel co-training scheme by exploiting complementary information from multiple views (time view and frequency view) of physiological signals to mine more positive samples, which overcomes the drawback of popular InfoNCE and achieves semantic-level representation learning; (2) we extend our framework with a memory module, implemented by a queue and a moving-averaged encoder, to enlarge the pool of negative candidates and keep the up-to-date representation; (3) extensive experiments conducted on sleep stage classification demonstrate state-of-the-art performance compared with SSL baselines, achieving 71.6% and 57.9% accuracies on two sleep datasets, SleepEDF and ISRUC respectively. The code is publicly available athttps://github.com/larryshaw0079/CoSleep.

The purpose of this paper is to learn efficient representations from raw electroencephalogram (EEG) signals for sleep stage classification via self-supervised learning (SSL). Although supervised methods have gained favorable performance, they heavily rely on manually labeled datasets. Recently, SSL arrives comparable performance with fully supervised methods despite limited labeled data by extracting high-level semantic representations. To alleviate the severe reliance of labels, we propose SleepDPC, a novel sleep stage classification algorithm based on SSL. By incorporating two dedicated predictive and discriminative learning principles, SleepDPC discovers underlying semantics from raw EEG signals in a more efficient manner. We thoroughly evaluate the performance of our proposed method on two publicly available datasets. The experimental results show that our method not only learns meaningful representations but also produces superior performance versus various competing methods despite limited access of labeled data.

Sleep staging is fundamental for sleep assessment and disease diagnosis. Although previous attempts to classify sleep stages have achieved high classification performance, several challenges remain open: 1) How to effectively extract salient waves in multimodal sleep data; 2) How to capture the multi-scale transition rules among sleep stages; 3) How to adaptively seize the key role of specific modality for sleep staging. To address these challenges, we propose SalientSleepNet, a multimodal salient wave detection network for sleep staging. Specifically, SalientSleepNet is a temporal fully convolutional network based on the $U^2$-Net architecture that is originally proposed for salient object detection in computer vision. It is mainly composed of two independent $U^2$-like streams to extract the salient features from multimodal data, respectively. Meanwhile, the multi-scale extraction module is designed to capture multi-scale transition rules among sleep stages. Besides, the multimodal attention module is proposed to adaptively capture valuable information from multimodal data for the specific sleep stage. Experiments on the two datasets demonstrate that SalientSleepNet outperforms the state-of-the-art baselines. It is worth noting that this model has the least amount of parameters compared with the existing deep neural network models.

The research on human emotion under multimedia stimulation based on physiological signals is an emerging field and important progress has been achieved for emotion recognition based on multi-modal signals. However, it is challenging to make full use of the complementarity among spatial-spectral-temporal domain features for emotion recognition, as well as model the heterogeneity and correlation among multi-modal signals. In this paper, we propose a novel two-stream heterogeneous graph recurrent neural network, named HetEmotionNet, fusing multi-modal physiological signals for emotion recognition. Specifically, HetEmotionNet consists of the spatial-temporal stream and the spatial-spectral stream, which can fuse spatial-spectral-temporal domain features in a unified framework. Each stream is composed of the graph transformer network for modeling the heterogeneity, the graph convolutional network for modeling the correlation, and the gated recurrent unit for capturing the temporal domain or spectral domain dependency. Extensive experiments on two real-world datasets demonstrate that our proposed model achieves better performance than state-of-the-art baselines.

Motor imagery (MI), as one of the important applications of brain-computer interface (BCI), has lately received great attention. However, current MI researches have not provided satisfactory representations of electroencephalogram (EEG), taking account of the space-time-frequency features for MI classification. Moreover, those models also lack the exploration of attentive spatial, temporal, and spectral dynamics. In this study, we propose TA3D (Two-stream Attention based 3D network), a novel model for MI classification. It mainly consists of two streams: the space-time stream and the space-frequency stream, representing and learning discriminative features in the space-time-frequency dimension. Specifically, each stream contains three key parts: 1) 3D representations of EEG signals depict the spatial information over temporal/spectral distributions; 2) Attention mechanisms adaptively explore attentive dynamics of EEG signals and focus on the most valuable information in separate dimensions; 3) 3D convolutions learn spatial representation, temporal dependence, and spectral dependence. The outputs of the two streams are concatenated for space-time-frequency feature fusion. Extensive experiments implemented on two BCI datasets demonstrate that our model outperforms state-of-the-art MI classification methods.

Sleep stage classification is essential for sleep assessment and disease diagnosis. However, how to effectively utilize brain spatial features and transition information among sleep stages continues to be challenging. In particular, owing to the limited knowledge of the human brain, predefining a suitable spatial brain connection structure for sleep stage classification remains an open question. In this paper, we propose a novel deep graph neural network, named GraphSleepNet, for automatic sleep stage classification. The main advantage of the GraphSleepNet is to adaptively learn the intrinsic connection among different electroencephalogram (EEG) channels, represented by an adjacency matrix, thereby best serving the spatial-temporal graph convolution network (ST-GCN) for sleep stage classification. Meanwhile, the ST-GCN consists of graph convolutions for extracting spatial features and temporal convolutions for capturing the transition rules among sleep stages. Experiments on the Montreal Archive of Sleep Studies (MASS) dataset demonstrate that the GraphSleepNet outperforms the state-of-the-art baselines.

Multimedia stimulation of brain activities has not only become an emerging field for intensive research, but also achieves important progress in the electroencephalogram (EEG) emotion classification based on brain activities. However, how to make full use of different EEG features and the discriminative local patterns among the features for different emotions is challenging. Existing models ignore the complementarity among the spatial-spectral-temporal features and discriminative local patterns in all features, which limits the classification ability of the models to a certain extent. In this paper, we propose a novel spatial-spectral-temporal based attention 3D dense network, named SST-EmotionNet, for EEG emotion recognition. The main advantage of the SST-EmotionNet is the simultaneous integration of spatial-spectral-temporal features in a unified network framework. Meanwhile, a 3D attention mechanism is designed to adaptively explore discriminative local patterns. Extensive experiments on two real-world datasets demonstrate that the SST-EmotionNet outperforms the state-of-the-art baselines.

In this paper, we propose a new automatic sleep stage classification method based on convolutional neural networks (CNN) using single-channel electroencephalogram (EEG). The sleep stages usually consist of five stages: awake, rapid eye movement (REM), and three non-rapid eye movement stages (N1/N2/SWS). In this method, we introduced two novel ways to convert EEG time series to meaningful matrices which CNN can handle with: the dynamical time-frequency spectrum based on Hilbert-Huang transform and the temporal feature matrix, capturing the characteristics of different sleep stages. Thirtynine whole-night sleep EEG signals from Physionet database were used to evaluate the performance of our proposed method. The results show that our proposed method give good classifications for most sleep stages, especially for awake and the SWS stages. Moreover, we obtain an average accuracy of 84.5%, which outperforms other four existing methods. Different from traditional methods based on artificial feature extracting, this method are simple and more applicable to various time series.