Recent research on Self-Supervised Learning (SSL) has demonstrated its ability to extract high-quality representations from unlabeled samples. However, in continual learning scenarios where training data arrives sequentially, SSL’s performance tends to deteriorate. This study focuses on Continual Contrastive Self-Supervised Learning (CCSSL) and highlights that the absence of inter-task contrastive learning, due to the unavailability of historical samples, leads to a significant drop in performance. To tackle this issue, we introduce a simple and effective method called BGE, which Bridges the inter-task Gap of CCSSL using External data from publicly available datasets. BGE enables the contrastive learning of each task data with external data, allowing relationships between them to be passed along the tasks, thereby facilitatingimplicitinter-task data comparisons. To overcome the limitation of the external data selection and maintain its effectiveness, we further propose the One-Propose-One algorithm to collect more relevant and diverse high-quality samples from external sources while filtering out distractions from the out-of-distribution data. Experiments show that BGE can generate better discriminative representation in CCSSL, especially for inter-task data, and improve classification results with various external data compositions. Additionally, BGE can be seamlessly integrated into existing continual learning methods, yielding significant performance improvement.

Continual learning in computer vision faces the critical challenge of catastrophic forgetting, where models struggle to retain prior knowledge while adapting to new tasks. Although recent studies have attempted to leverage the generalization capabilities of pre-trained models to mitigate overfitting on current tasks, models still tend to forget details of previously learned categories as tasks progress, leading to misclassification. To address these limitations, we introduce a novel Knowledge Graph Enhanced Generative Multi-modal model (KG-GMM) that builds an evolving knowledge graph throughout the learning process. Our approach utilizes relationships within the knowledge graph to augment the class labels and assigns different relations to similar categories to enhance model differentiation. During testing, we propose a Knowledge Graph Augmented Inference method that locates specific categories by analyzing relationships within the generated text, thereby reducing the loss of detailed information about old classes when learning new knowledge and alleviating forgetting. Experiments demonstrate that our method effectively leverages relational information to help the model correct mispredictions, achieving state-of-the-art results in both conventional CIL and few-shot CIL settings, confirming the efficacy of knowledge graphs at preserving knowledge in the continual learning scenarios.

Class Incremental Learning (CIL) for image classification aims to address real-world scenarios by allowing a model to learn new categories while retaining the knowledge of old categories. It is more challenging than Task Incremental Learning (TIL) as task ID is not provided during testing. Therefore, transitioning from CIL to TIL is an intuitive approach to handling CIL problems for image classification. Currently, the main challenge of this approach lies in improving the accuracy of task identification. To address this issue, we propose to use a large-scale image-text pre-training model (i.e. CLIP) as the backbone, training and saving different classifiers for different tasks. Each classifier not only includes the classes of the current task, but also an Out-of-distribution (OOD) class corresponding to the classes encountered in all previous tasks. At test time, we iterate through classifiers from the last task to find the correct task ID of the test image, and perform classification in a TIL way. In addition, to tackle the issue of early-stop termination in iterative prediction due to model bias toward later tasks, we propose using CLIP zero-shot ability to assist learned OOD detection. Experiments show that our method achieves state-of-the-art performance on the traditional many-shot and the more challenging few-shot settings of CIFAR-100 and ImageNet-Subset datasets.

In class-incremental learning (CIL) scenarios, the phe-nomenon of catastrophic forgetting caused by the classi-fier's bias towards the current task has long posed a signif-icant challenge. It is mainly caused by the characteristic of discriminative models. With the growing popularity of the generative multi-modal models, we would explore replacing discriminative models with generative ones for CIL How-ever, transitioning from discriminative to generative mod-els requires addressing two key challenges. The primary challenge lies in transferring the generated textual infor-mation into the classification of distinct categories. Ad-ditionally, it requires formulating the task of CIL within a generative framework. To this end, we propose a novel generative multi-modal model (GMM) framework for class-incremental learning. Our approach directly generates la-bels for images using an adapted generative model. After obtaining the detailed text, we use a text encoder to ex-tract text features and employ feature matching to deter-mine the most similar label as the classification prediction. In the conventional CIL settings, we achieve signifi-cantly better results in long-sequence task scenarios. Un-der the Few-shot CIL setting, we have improved by at least 14% accuracy over all the current state-of-the-art methods with significantly less forgetting. Our code is available at https://github.com/DoubleClass/GMM.