Multimodal emotion recognition has garnered significant attention for its ability to integrate data from multiple modalities to enhance performance. However, physiological signals like electroencephalogram are more challenging to acquire than visual data due to higher collection costs and complexity. This limits the practical application of multimodal networks. To address this issue, this paper proposes a cross-modal knowledge distillation framework for emotion recognition. The framework aims to leverage the strengths of a multimodal teacher network to enhance the performance of a unimodal student network using only the visual modality as input. Specifically, we design a prototype-based modality rebalancing strategy, which dynamically adjusts the convergence rates of different modalities to mitigate modality imbalance issue. It enables the teacher network to better integrate multimodal information. Building upon this, we develop a Cross-Modal Densely Guided Knowledge Distillation (CDGKD) method, which effectively transfers knowledge extracted by the multimodal teacher network to the unimodal student network. Our CDGKD uses multi-level teacher assistant networks to bridge the teacher-student gap and employs dense guidance to reduce error accumulation during knowledge transfer. Experimental results demonstrate that the proposed framework outperforms existing methods on two public emotion datasets, providing an effective solution for emotion recognition in modality-constrained scenarios.

Accurate sleep staging is crucial for the diagnosis of diseases such as sleep disorders. Existing sleep staging models with excellent performance are usually large and require a lot of computational resources, limiting their application on wearable devices. Therefore, it is a key issue to distil the knowledge embedded in large models into small heterogeneous models for better deployment. In the process of knowledge distillation of heterogeneous models for sleep electroencephalography (EEG) signals, we mainly deal with three major challenges: 1) There are large structural differences between heterogeneous sleep staging models; 2) What kind of knowledge should be conveyed in sleep EEG signals in the knowledge distillation of heterogeneous models; 3) Significant scale differences exist between heterogeneous models. To address these challenges, we design a generic heterogeneous model knowledge distillation framework for sleep staging. Specifically, we first propose a knowledge distillation strategy for heterogeneous models that addresses the large structural differences between heterogeneous models. Then, a multi-level knowledge distillation module is designed to effectively transfer important multi-level feature knowledge. In addition, the teacher assistant module is introduced to ease the scale difference between the heterogeneous models which further enhances the knowledge distillation performance. Experimental results on both Sleep-EDF and ISRUC datasets show that our distillation framework achieves state-of-the-art performance.

Sleep stage classification is of great significance to the diagnosis of sleep disorders. However, existing sleep stage classification models based on deep learning are usually relatively large in size (wider and deeper), which makes them hard to be deployed on wearable devices. Therefore, it is a challenge to lighten the existing sleep stage classification models. In this paper, we propose a novel general knowledge distillation framework for sleep stage classification tasks called SleepKD. Our SleepKD, composed of the multi-level module, teacher assistant module, and other knowledge distillation modules, aims to lighten large-scale sleep stage classification models. Specifically, the multi-level module is able to transfer the multi-level knowledge extracted from sleep signals by the teacher model (large-scale model) to the student model (lightweight model). Moreover, the teacher assistant module bridges the large gap between the teacher and student network, and further improves the distillation. We evaluate our method on two public sleep datasets (Sleep-EDF and ISRUC-III). Compared to the baseline methods, the results show that our knowledge distillation framework achieves state-of-the-art performance. SleepKD can significantly lighten the sleep model while maintaining its classification performance. The source code is available at https://github.com/HychaoWang/SleepKD.