With the continuous rise in the prevalence of skin diseases, accurate classification has become an urgent demand in clinical diagnosis and treatment. In recent years, deep learningbased methods have demonstrated excellent performance in skin lesion classification tasks. To further improve diagnostic accuracy, multimodal fusion approaches integrating dermoscopic images, clinical photographs, and metadata have been explored. However, many existing methods still fail to effectively capture the deep semantic associations among these modalities. This oversight leads to semantic fragmentation across modalities, loss of critical diagnostic cues, and ultimately hinders classification accuracy. To address this challenge, we propose a Metadata-Guided Multimodal Consistency Constraints (MG-MCC) framework, which leverages metadata as a semantic anchor and enforces collaborative constraints through three specially designed loss functions: Metadata-Aware Alignment (MA), Lesion Feature Purification (LFP), and Semantic Consistency Preservation (SCP). Specifically, the MA loss enforces precise cross-modal alignment between image features and metadata in a unified space; the LFP loss strengthens semantic associations between core lesion features and metadata to accurately separate background features; and the SCP loss avoids the loss of diagnostic ROI information caused by Instance Normalization. Extensive experiments conducted on two public skin lesion datasets demonstrate that the proposed MG-MCC framework significantly outperforms state-of-the-art methods in skin lesion classification tasks.

Integrating multimodal learning (ML) with polysomnography (PSG) has emerged as a research hotspot for reliable sleep staging. However, the complexity of these signals and the discomfort associated with wearing multi-lead devices somewhat limit the feasibility of daily and ubiquitous sleep monitoring. Unfortunately, most existing ML paradigms are constrained by consistent and fixed input patterns. When the number of modalities is less than required by the ML framework, it is easy to cause inference bias, resulting in a significant performance degradation. To this end, we propose an elastic multimodal sleep staging network (ElaSleepNet), consisting of multimodal information completion (MIC) and adaptive cross-modal (ACM) interaction. Specifically, MIC maximizes the consistency of multimodal signals on intra-epoch temporal level and inter-epoch contextual level, thereby enhancing the reasoning and completion abilities of the available modalities for the unavailable modalities. Moreover, we introduce learnable parameters and design the ACM attention mechanism, which allows handling multimodal information interaction while maintaining robustness in the absence of certain modalities. Our ElaSleepNet demonstrates its state-of-the-art on three multimodal sleep datasets. Compared with previous methods, ElaSleepNet can achieve better performance with fewer testing modalities, making it flexible for daily monitoring.

Multimodal physiological signals, such as EEG, EOG and EMG, provide rich and reliable physiological information for automated sleep staging (ASS). However, in the real world, the completeness of various modalities is difficult to guarantee, which seriously affects the performance of ASS based on multimodal learning. Furthermore, the exploration of temporal context information within PTSs is also a serious challenge. To this end, we propose a robust multimodal sleep staging framework named contrastive imagination modality sleep network (CIMSleepNet). Specifically, CIMSleepNet handles the issue of arbitrary modal missing through the combination of modal awareness imagination module (MAIM) and semantic & modal calibration contrastive learning (SMCCL). Among them, MAIM can capture the interaction among modalities by learning the shared representation distribution of all modalities. Meanwhile, SMCCL introduces prior information of semantics and modalities to check semantic consistency while maintaining the uniqueness of each modality. Utilizing the calibration of SMCCL, the data distribution recovered by MAIM is aligned with the real data distribution. We further design a multi-level cross-branch temporal attention mechanism, which can facilitate the mining of interactive temporal context representations at both the intra-epoch and inter-epoch levels. Extensive experiments on five multimodal sleep datasets demonstrate that CIMSleepNet remarkably outperforms other competitive methods under various missing modality patterns. The source code is available at: https://github.com/SQAIYY/CIMSleepNet.