Chen Xu, Yu ji, Zhenyu Lv, Yang Yi, Yizhe Yang, Luyao Ji, Chaoyi Chen, Xianyang Wang, Tian Lan, Zhihua Wang, Juan Wang, Xunde Dong, Fuze Tian, Qunxi Dong, Bin Hu. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Current multimodal sentiment analysis (MSA) and emotion recognition in conversations (ERC) methods based on pre-trained language models exhibit two primary limitations: 1) Once trained for MSA and ERC tasks, these pre-trained language models lose their original generalized capabilities. 2) They demand considerable computational resources. As the size of pre-trained language models continues to grow, training larger multimodal sentiment analysis models using previous approaches could result in unnecessary computational cost. In response to this challenge, we propose Multimodal Sentiment Analysis and Emotion Recognition Adapter (MSE-Adapter), a lightweight and adaptable plugin. This plugin enables a large language model (LLM) to carry out MSA or ERC tasks with minimal computational overhead (only introduces approximately 2.6M to 2.8M trainable parameters upon the 6/7B models), while preserving the intrinsic capabilities of the LLM. In the MSE-Adapter, the Text-Guide-Mixer (TGM) module is introduced to establish explicit connections between non-textual and textual modalities through the Hadamard product. This allows non-textual modalities to better align with textual modalities at the feature level, promoting the generation of higher-quality pseudo tokens. Extensive experiments were conducted on four public English and Chinese datasets using consumer-grade GPUs and open-source LLMs (Qwen-1.8B, ChatGLM3-6B-base, and LLaMA2-7B) as the backbone. The results demonstrate the effectiveness of the proposed plugin.

Automated electrocardiogram (ECG) report generation provides clinicians with interpretable diagnostic evidence, which is of great significance in enhancing the clinical acceptability of ECG intelligent diagnosis. Retrieval-Augmented Generation (RAG) effectively mitigates factual hallucinations in large models through the incorporation of external medical knowledge. However, current RAG-based approaches suffer from drawbacks like insufficient retrieval relevance and limited flexibility in ECG report generation, thus failing to satisfy the clinical requirements for report accuracy and applicability. To address these challenges, this study presents a dual-conditional RAG method, LGFDTK-RAG, which is based on Label-Guided Filtering (LGF) and Dynamic Top-K (DTK), for automated ECG report generation. The LGF mechanism is designed to select retrieval contexts that are consistent with ECG diagnostic labels, thereby improving the clinical consistency of the generated reports. The DTK strategy, which is based on the marginal utility theory, dynamically optimizes the number of retrieval contexts (i.e., the$K$value). This approach not only ensures the coverage of key information but also effectively reduces redundancy. Based on the above-mentioned mechanisms, a small language model (SLM) is utilized to generate ECG reports. Experiments conducted on the PTB-XL dataset indicate that LGFDTK-RAG enhances both the clinical consistency of the generated reports and the comprehensiveness of diagnostic information, all the while maintaining a relatively low consumption of computational resources.

Electrocardiogram (ECG) is a critical non-invasive diagnostic tool for monitoring clinical cardiac conditions. Most research on ECG classification based on deep learning has made significant progress, but interpretability remains a major challenge, leading to lower acceptance by cardiologists. Moreover, most existing studies focus on analyzing ECG signal, neglecting the fact that cardiologists primarily make diagnoses based on waveform changes displayed in ECG image. Therefore, this study proposes a method called BiECG-LLM, which integrates information from Bi-modal ECG (i.e., ECG signal and image) and utilizes a fine-tuned Large Language Model (LLM) to achieve ECG classification and report generation. Specifically, we presented a Bi-modal Interactive Guided Fusion (Bi-IGF) module to align and fuse bi-modal information, enhancing the integration of ECG signal and image features in the latent space. This significantly improves the LLM's ability to interpret bi-modal ECG, making ECG classification and report generation more accurate. Most importantly, the generated reports include descriptions of ECG waveform morphology, enhancing the interpretability of ECG classification. Extensive experimental results on three large databases demonstrate the superiority of BiECG-LLM. The code will be released on GitHub after a blind review.

Automated ECG analysis researches can be categorized into single-lead and multi-lead approaches. While multi-lead ECG diagnostic models have achieved expert-level performance using deep learning, their data acquisition is less convenient compared to single-lead ECGs, limiting their application. Additionally, traditional knowledge distillation techniques face challenges in handling multi-label data. To enhance multi-label single-lead ECG-based cardiac disease diagnosis, this paper presents DF-MLSL, an efficient knowledge distillation frame-work designed for multi-label single-lead ECG classification. The framework consists of three main losses: Last Layer Features Distillation (LLFD) based on feature matching, Sigmoid Regression Distillation (SRD) loss for decoupled representation learning and classification, and Multi-Label Logits Distillation (MLLD) loss designed for multi-label data. In this framework, the teacher network (T-net) is pretrained. Subsequently, the student network (S-net) is trained based on a composite loss function that incorporates three specialized losses along with standard binary cross-entropy (BCE) loss. This enables the transfer of latent knowledge from the T-net to the S-net. Experimental evaluations were conducted on three multi-label datasets to demonstrate the effectiveness of DF-MLSL in enhancing the performance of the S-net for single-lead multi-label ECG classification tasks.

In the field of automatic Electrocardiogram (ECG) diagnosis, due to the relatively limited amount of labeled data, how to build a robust ECG pretrained model based on unlabeled data is a key area of focus for researchers. Recent advancements in contrastive learning-based ECG pretrained models highlight the potential of exploiting additional patient-level selfsupervisory signals inherent in ECG. They are referred to as patient contrastive learning. Its rationale is that multiple physical recordings from the same patient may share commonalities, termed patient consistency, so redefining positive and negative pairs in contrastive learning as intra-patient and inter-patient samples provides more shared context to learn an effective representation. However, these methods still fail to efficiently exploit patient consistency due to the insufficient amount of intraand inter-patient samples existing in a batch. Hence, we propose a contrastive learning-based ECG pretrained model enhanced by the Patient Memory Queue (PMQ), which incorporates a large patient memory queue to mitigate model degeneration that can arise from insufficient intra- and inter-patient samples. In order to further enhance the performance of the pretrained model, we introduce two extra data augmentation methods to provide more perspectives of positive and negative pairs for pretraining. Extensive experiments were conducted on three public datasets with three different data ratios. The experimental results show that the comprehensive performance of our method outperforms previous contrastive learning methods and exhibits greater robustness in scenarios with limited labeled data. The code is available at Github.

Emotion recognition holds significant value in fields such as mental health monitoring, human-computer interaction enhancement, and behavioral cognition research. ECG-based methods have advantages in resisting subjective masking. However, current ECG-based emotion recognition methods typically partition ECG signals into small segments for analysis, and then randomly divide them to obtain training, test, and validation sets. This may not only cut off the continuous changes in ECG caused by emotional changes, affecting the recognition performance, but also lead to data leakage due to the random division of the dataset, seriously affecting the model's generalization ability. Moreover, existing studies usually recognize emotions in a single dimension (Arousal or Valence), making it difficult to comprehensively depict subjects' emotional states. To address these issues, this paper first jointly defines two-dimensional emotion labels by combining arousal and valence and divides them into four quadrants. Second, it divides the training, test, and validation sets in a subject-independent manner based on statistical data. In particular, this paper proposes a lightweight HiSAN framework that can process long-sequence raw ECG signals. This framework extracts features through Grouped Convolution (GC) blocks, then uses the Hierarchical Segment Attention (HiSA) mechanism to capture multi-scale emotional features, and finally applies them to emotion recognition. Extensive experimental results on three public datasets (AMIGOS, DREAMER, and SWELL) show that HiSAN has superior emotion recognition performance and low computational consumption. In particular, this study can serve as a benchmark for future related research.

Emotion recognition has important application value in fields such as mental health monitoring, education, and medical diagnosis. Researchers have widely explored emotion recognition research based on electrocardiogram (ECG). Nonetheless, current methods typically predict emotional valence or arousal independently, hindering a comprehensive portrayal of an individual's overall emotional state. Moreover, ECG signals are commonly segmented and randomly split. This can lead to incomplete coverage of emotion-related periods in ECG segments and may also cause data leakage in the dataset. Based on the circumplex theory of emotion, this study maps emotional valence and arousal onto a four-quadrant emotional space. Subsequently, the AMIGOS, DREAMER, and SWELL datasets are partitioned into training, validation, and test sets following the inter-subject paradigm. On this basis, we propose a lightweight emotion recognition model, EmoNet-ECG, which includes a Stem layer, an Efficient Local Feature Compression (ELF C) module, a Depth-wise Attention-Replacing (DAR) module, and a Convolution and Attention (CA) module, forming an inverted pyramid feature extraction framework. In particular, to compensate for the loss of degradation information, we design a long-range shortcut connection. Extensive experimental results show that EmoNet-ECG has superior emotion recognition ability and generalization performance.

Myocardial infarction (MI) is a prevalent and serious cardiovascular condition. The 12-lead electrocardiogram (ECG) is essential for MI diagnosis, as it reveals unique electrical patterns from different heart locations. Accurate localization and assessment of MI require a comprehensive analysis of ECG signals from multiple views. However, previous researches have primarily analyzed the 12-lead ECG from a single view, neglecting the variations in MI localization across different leads. Therefore, this study proposes a Multi-View Teacher-Student Framework (MV-TSF) for MI localization, which integrates multi-view learning and knowledge distillation. MV-TSF consists of two sub-networks: Multi-View Teacher network (MVT-net) and Single-View Student network (SVS-net). MVT-net treats the 12-lead ECG as five distinct views based on the correspondence between different heart regions and leads, and SVS-net uses the 12-lead ECG as input. Both sub-networks employ a multi-layer convolutional neural network structure for varied-scale ECG feature extraction. Additionally, MVT-net introduces an effective method for merging feature vectors from different views. Through knowledge distillation, knowledge is transferred from MVT-net to SVS-net, resulting in a Distilled SVS-net (DSVS-net) with only 0.35M parameters. Experimental results on two multi-label datasets indicate that DSVS-net is highly competitive, demonstrating exceptional parameter efficiency, inference speed, and model performance.

Most existing Multimodal Sentiment Analysis (MSA) models that are enhanced with external sentiment knowledge primarily focus on integrating this knowledge during the multimodal fusion stage, while overlooking its potential benefits during the multimodal representation learning process. In this study, we propose a Sentiment Knowledge-Enhanced Multimodal Sentiment Analysis (SKE-MSA) framework, which incorporates an external sentiment knowledge base—the VAD lexicon—to enhance MSA. SKE-MSA introduces the Sentiment-aware Encoding Loss (SAE_Loss), designed to leverage external sentiment knowledge to guide the representation learning of multimodal encoders. Building on this, we extract both common and unique features of multimodal representations and subsequently predict sentiment intensity based on these features. Extensive experiments conducted on three MSA datasets demonstrate the competitive performance of SKE-MSA.

Electrocardiogram (ECG) is essential for diagnosing cardiovascular diseases. The Transformer architecture, employing multi-head self-attention, has shown promise in ECG classification due to its strong sequence modeling capabilities. However, Transformers suffer from suboptimal inference efficiency. State Space Models (SSMs) have garnered attention for their efficient inference. This study introduces ECGMamba, a novel ECG classification model based on Mamba, a specialized SSM. ECGMamba comprises the ECG encoder and the Mamba layer centered on Mamba. The encoder uses three one-dimensional convolutions to extract ECG local information. The core of the Mamba layer is the Multi-Path Mamba-based block, which models global contextual semantic information. This architecture ensures efficient inference and significant performance improvements. Experimental results on PTB-XL and CPSC2018 databases demonstrate ECGMamba’s competitive performance and efficient inference capabilities. This study suggests a new approach for efficient and accurate ECG classification.

Cardiac arrhythmia is a common cardiovascular disease that can cause sudden death in severe cases. Electro-cardiography (ECG) is the most well-known and widely applied method for heart diseases detection. Computer-aided diagnosis of ECG can help improve physician efficiency and reduce the rate of misdiagnosis of ECG. In this paper, we propose a method for arrhythmia classification based on the dense convolutional network (DenseNet) and efficient channel attention (ECA). Evaluation experiments were performed using the ECG records from the MIT-BIH database. The accuracy, sensitivity, specificity, and F1 values of 99.69%, 97.55%, 99.81%, and 97.72% were achieved for the six types of heartbeats classification, respectively. The experimental results demonstrate the validity and feasibility of the method, which can be used for ECG screening.

Automatic ECG beat classification plays an important role in detecting cardiac disease. In this paper, we propose an automatic recognition model for ECG signals based on discrete wavelet transform (DWT), principal component analysis (PCA), kernel principal component analysis (KPCA), and adaptive kernel principal component analysis (AKPCA). We extracted different ECG features using DWT, PCA, KPCA, and AKPCA, respectively. These features were combined and used as support vector machine (SVM) input to classify the ECG. ECG records taken from the MIT-BIH arrhythmia database are selected to test the proposed method. The following five heartbeat types were classified using this method: normal beats (N), premature ventricular beats (V), right bundle branch block beats (R), left bundle branch block beats (L), and premature atrial beats (A). The sensitivity, accuracy, precision, and specificity reached 99.95%, 99.86%, 99.53%, and 99.70%, respectively. These results indicate the proposed method is reliable and efficient for ECG beat classification.