Vision-Language Continual Learning (VLCL) has attracted significant research attention for its robust capabilities, and the adoption of Parameter-Efficient Fine-Tuning (PEFT) strategies is enabling these models to achieve competitive performance with substantially reduced resource consumption. However, dominated First-Order (FO) optimization is prone to trap models in suboptimal local minima, especially in limited exploration subspace within PEFT. To overcome this challenge, this paper pioneers a systematic exploration of adopting Zeroth-Order (ZO) optimization for PEFT-based VLCL. We first identify the incompatibility of naive full-ZO adoption in VLCL due to optimization process instability. We then investigate the application of ZO optimization from a modality branch-wise to a fine-grained layer-wise across various training units to identify an optimal strategy. Besides, a key theoretical insight reveals that vision modality exhibit higher variance than language counterparts in VLCL during the ZO optimization process, and we propose a modality-aware stabilized ZO strategy, which adopts gradient sign normalization in ZO and constrains vision modality perturbation to further improve performance. Benefiting from the adoption of ZO optimization, PEFT-based VLCL fulfills better ability to escape local minima during the optimization process, extensive experiments on four benchmarks demonstrate that our method achieves state-of-the-art results.

The primary goal of change captioning is to identify subtle visual differences between two similar images and express them in natural language. Existing research has been significantly influenced by the task of vision change detection and has mainly concentrated on the identification and description of visual changes. However, we contend that an effective change captioner should go beyond mere detection and description of what has changed. Two additional aspects are crucial: 1) retaining significant and unique semantic elements that persist across both images, and 2) forging a robust link between visual cues and their concomitant descriptive linguistic elements. This paper addresses these challenges by presenting the C 3 aptioner, which seamlessly incorporates dual momentum contrastive learning objectives into change captioning. Our model architecture consists of intra-image and inter-image Transformer encoders for visual feature extraction, complemented by unimodal language and multimodal decoders. Specifically, we introduce a cross-view contrastive learning objective to capture essential invariant features by aligning cross-view representations with a momentum-updated queue of negative samples, addressing the challenge of viewpoint variations. Additionally, our cross-modality contrastive learning objective aligns and interacts visual and textual modalities using a separate momentum-maintained queue, resolving the modality gap that hampers existing methods. This dual contrastive approach enables C 3 aptioner to model both changed and unchanged elements while establishing strong vision-language correspondence, resulting in more contextually rich and human-like descriptions. Extensive experiments across five distinct datasets confirm that our approach achieves state-of-the-art performance, with particularly significant improvements in challenging scenarios involving extreme viewpoint changes. Source code is available at https://github.com/DenglinGo/C-3aptioner .

Whole Slide Imaging (WSI) has become a gold standard in cancer diagnosis, inspecting multi-scale information from cellular to tissue levels. Processing an entire WSI directly is infeasible due to GPU memory constraints; thus, Multiple Instance Learning (MIL) has emerged as the standard solution by partitioning WSIs into tiles. While recent two-stage MIL frameworks partially achieve memory efficiency by decoupling tile-level extraction from slide-level modeling, they still face four limitations: 1) the conflict between training throughput and inference memory efficiency, 2) the high susceptibility to overfitting on small-scale WSI datasets with sparse supervision, 3) the disruption of spatial structural integrity during sampling-based training, and 4) the inadequate modeling of multi-scale feature interactions within long sequences. We therefore introduce PathRWKV, a novel State Space Model designed for efficient and robust WSI analysis. To resolve the computational trade-off, we propose an asymmetric structure utilizing max pooling aggregation, enabling parallelized training for high throughput and recurrent inference with constant ( $\mathcal {O}\text {(}{1}\text {)}$ ) memory complexity. To mitigate overfitting, we employ random sampling to enhance data diversity, with a multi-task learning module to regularize feature learning on limited data. To restore spatial context, we introduce 2D sinusoidal position encoding to perceive the relative locations of tissue tiles. To capture comprehensive representations, we integrate TimeMix and ChannelMix modules, enabling dynamic multi-scale feature modeling across temporal and spatial dimensions. Experiments on 29,073 WSIs across 11 datasets demonstrate that PathRWKV outperforms 11 state-of-the-art methods on 10 datasets, establishing it as a scalable and solution with application potential.