Web-scale content is rich in modalities yet frequently incomplete due to device limits, transmission errors, or privacy controls, making learning with missing modalities a core challenge. Prior reconstruction and alignment strategies often fail to preserve a stable class geometry when inputs are partial, leading to anchor drift -- a shift of class prototypes between complete and incomplete views that distorts the shared representation space and degrades generalization. We introduce HiCoD (Hierarchical Consistency-Guided Pro mpt Distillation), which learns a robust, class-anchored semantic space. HiCoD combines: (1) a modality-aware semantic graph that restores cross-modal structure under partial observations; (2) dual-level anchoring that unifies large-language-model–derived global category prototypes with top-K local exemplars to balance cross-modal coherence and modality-specific detail; and (3) multi-level distillation that aligns unimodal features, fused embeddings, and prompt-completed signals within a single anchor space. Across CMU-MOSI, CMU-MOSEI, and additional benchmarks, HiCoD sets a new state of the art under both fixed-pattern and random missingness, improving Acc-2 by up to 6.4 points over MPLMM and remaining robust when key modalities are absent.

Recently, prompt tuning has garnered considerable attention due to its success across various Vision-Language (VL) tasks. However, unimodal prompts, coupled prompts, and joint prompts in these models often lead to suboptimal performance due to differences in information density and complexity between modalities. Particularly, in scenarios with missing modalities, these prompt-based approaches tend to exacerbate 'Channel Bias'-a phonomenon where models overly rely on specific feature (such as unmissing-modal feature) channels from the base tasks, thereby undermining the model's ability to capture crucial shared knowledge applicable to new tasks and affecting its generalizability. To address this challenge, we propose RobustPT, a dynamic disentanglement prompt tuning model designed to enhance the robustness of VL models under modality missing conditions. RobustPT utilizes a multi-channel prompting mechanism to dynamically disentangle and align prompts. Specifically, RobustPT is divided into single-channel tuning and alignment-channel tuning, where prompts for each modality run independently in sequence to delve deeply into their intrinsic characteristics, followed by an integration through a non-strong coupling strategy to effectively balance information contributions and enhance overall performance. Extensive experiments demonstrate that our RobustPT achieve significant improvements over the current state-of-the-art across all benchmark datasets. Our codes are available at https://github.com/Trae1ounG/RobustPT.

Multimodal learning in open environments faces significant challenges due to modality incompleteness and noise interference. Current prompt engineering emphasizes modality absence over task-instance contextualization that hinders cross-modal knowledge transfer, whereas conditional generation approaches for missing modality recovery exhibit an excessive reliance on the quality of available modalities. To address these issues, we propose a Gamma-Cooperative Retrieval Diffusion model (γ-CRD), inspired by the human brain's multi-source contextual completion mechanism, which leverages a retrieval-augmented prompt generation framework and normal-inverse Gamma noise modeling to enhance robustness in incomplete multimodal learning. Specifically, it consists of three modules: (1) Retrieval-Augmented Contextualization: Construct a multimodal memory bank and retrieve relevant instances via similarity calculation under a gating mechanism to augment the contextualization of missing modalities. (2) Prompt-Driven Diffusion Generation Module: Builds prompts based on retrieval results and incorporates them into a denoising diffusion probabilistic model through an attention mechanism to enhance contextualized knowledge transfer and generate missing modalities. (3) Inverse-Gamma Noise Optimization Module: Model a mixed normal-inverse gamma distribution, which is dynamically aware of noise and enables uncertainty estimation in multimodal fusion, ensuring robust and reliable multimodal regression. Extensive experiments on three real-world datasets demonstrate that γ-CRD consistently outperforms state-of-the-art baselines, especially achieving a 5.7% improvement in accuracy compared to the leading model e.g., IMDer [1].

Commonsense question answering has demonstrated considerable potential across various applications like assistants and social robots.Although fully fine-tuned Pre-trained Language Model(PLM) has achieved remarkable performance in commonsense reasoning, their tendency to excessively prioritize textual information hampers the precise transfer of structural knowledge and undermines interpretability.Some studies have explored combining Language Models (LM) with Knowledge Graphs (KGs) by coarsely fusing the two modalities to perform Graph Neural Network (GNN)-based reasoning that lacks a profound interaction between heterogeneous modalities.In this paper, we propose a novel Graph-based Structure-Aware Prompt Learning Model for commonsense reasoning, named G-SAP, aiming to maintain a balance between heterogeneous knowledge and enhance the cross-modal interaction within the LM+GNNs model.In particular, an evidence graph is constructed by integrating multiple knowledge sources, i.e.ConceptNet, Wikipedia, and Cambridge Dictionary to boost the performance.Afterward, a structure-aware frozen PLM is employed to fully incorporate the structured and textual information from the evidence graph, where the generation of prompts is driven

Inferring geographic locations via social posts is essential for many practical location-based applications such as product marketing, point-of-interest recommendation, and infector tracking for COVID-19. Unlike image-based location retrieval or social-post text embedding-based location inference, the combined effect of multi-modal information (i.e., post images, text, and hashtags) for social post positioning receives less attention. In this work, we collect real datasets of social posts with images, texts, and hashtags from Instagram and propose a novel Multi-modal Representation Learning Framework (MRLF) capable of fusing different modalities of social posts for location inference. MRLF integrates a multi-head attention mechanism to enhance location-salient information extraction while significantly improving location inference compared with single domain-based methods. To overcome the noisy user-generated textual content, we introduce a novel attention-based character-aware module that considers the relative dependencies between characters of social post texts and hashtags for flexible multi-model information fusion. The experimental results show that MRLF can make accurate location predictions and open a new door to understanding the multi-modal data of social posts for online inference tasks.