Test-time adaptation (TTA) enables online model adaptation using only unlabeled test data, aiming to bridge the gap between source and target distributions. However, in multimodal scenarios, varying degrees of distribution shift across different modalities give rise to a complex coupling effect of unimodal shallow feature shift and cross-modal high-level semantic misalignment, posing a major obstacle to extending existing TTA methods to the multimodal field. To address this challenge, we propose a novel multimodal test-time adaptation (MMTTA) framework, termed as Bridging Modalities via Progressive Re-alignment (BriMPR). BriMPR, consisting of two progressively enhanced modules, tackles the coupling effect with a divide-and-conquer strategy. Specifically, we first decompose MMTTA into multiple unimodal feature alignment sub-problems. By leveraging the strong function approximation ability of prompt tuning, we calibrate the unimodal global feature distributions to their respective source distributions, so as to achieve the initial semantic re-alignment across modalities. Subsequently, we assign the credible pseudo-labels to combinations of masked and complete modalities, and introduce inter-modal instance-wise contrastive learning to further enhance the information interaction among modalities and refine the alignment. Extensive experiments on MMTTA tasks, including both corruption-based and real-world domain shift benchmarks, demonstrate the superiority of our method.

The domain shift between training and test data has emerged as a critical obstacle to the generalization of deep models. Continual Test-Time Adaptation (CTTA), which aims to leverage online test data stream to adapt to continuously evolving target distributions, has become a promising and practical solution to address real-world domain shift issues. Existing CTTA methods primarily rely on self-training frameworks based on output consistency and entropy-based loss functions, which often consider either the entire batch or reliable samples within the batch in isolation, failing to combine both of the advantages simultaneously. Inspired by fast and slow learning strategies in continual learning, we propose a fully source-free approach, dubbed CLAss-wise Shift-aware Hierarchical Continual Test-Time Adaptation (CLASH-CTTA). The adaptation process of CLASH-CTTA is carried out with a hierarchical updating strategy, where the test data stream promotes the slow learning of general representations, while the representative samples facilitate fast learning of domain-specific knowledge. The observation reveals that different classes exhibit varying sensitivities to different domains, leading to diverse discriminability across classes under domain shifts. Thus, a Class-wise Shift-aware Representative Set is maintained to provide representatives and mitigate the discrepancy. In addition, using Spearman's rank correlation as a novel perspective to examine the correctness of samples, we further filter representatives and align them to the class prototypes. Extensive experiments on three corruption domain shift datasets and one natural domain shift dataset demonstrate the superiority of our method compared with state-of-the-art methods, including in continual tasks, gradual tasks, and scenarios with diverse batch sizes.