Large multimodal language models (MLLMs) have revolutionized natural language processing and visual understanding, but often contain outdated or inaccurate information. Current multimodal knowledge editing evaluations are limited in scope and potentially biased, focusing on narrow tasks and failing to assess the impact on in-domain samples. To address these issues, we introduce ComprehendEdit, a comprehensive benchmark comprising eight diverse tasks from multiple datasets. We propose two novel metrics: Knowledge Generalization Index (KGI) and Knowledge Preservation Index (KPI), which evaluate editing effects on in-domain samples without relying on AI-synthetic samples. Based on insights from our framework, we establish Hierarchical In-Context Editing (HICE), a baseline method employing a two-stage approach that balances performance across all metrics. This study provides a more comprehensive evaluation framework for multimodal knowledge editing, reveals unique challenges in this field, and offers a baseline method demonstrating improved performance. Our work opens new perspectives for future research and provides a foundation for developing more robust and effective editing techniques for MLLMs.

Continual learning, focusing on sequential knowledge acquisition and retention, necessitates efficient memory management. This paper introduces a holistic approach, diverging from traditional methods that separately optimize neural network and replay buffer memory. We aim to enhance overall memory efficiency, addressing neural network parameters and replay buffer concurrently within strict memory constraints. This is achieved by harnessing neural network parameter redundancies and employing compression techniques like pruning and quantization, allowing data replay storage without extra memory overhead. Balancing memory use across components is challenging due to the complex search space of combined tasks. We tackle this by conceptualizing it as a bi-level optimization problem, integrating all tasks under a single objective, thus optimizing memory use and managing the interplay between different components. We employ a synergy of optimization techniques to solve this challenging bi-level optimization problem. Our experimental findings affirm the superior performance of our proposed method, outperforming existing techniques such as prompt-based, feature-replay, exemplar-replay, and regularization-based methods under stringent memory constraints, consistently across various datasets and neural network architectures.