Few-shot class-incremental learning (FSCIL) requires learning new knowledge without forgetting old knowledge. Forward compatibility can reserve space for novel classes while maintaining base class knowledge in incremental learning. Better forward compatibility is crucial for effectively mastering all knowledge, especially when dealing with a few unknown new classes. In this article, we propose the better forward compatibility pretraining (BFCP) to further enhance forward compatibility in FSCIL. We adopt a two-stage training for the backbone network in the base session. First, we train the backbone network at the image-level to enhance its feature extraction capability, enabling the model to extract valuable information from unknown class images. Second, we fine-tune the backbone network at the feature-level with fake prototypes and instances to achieve clustering base classes and reserve space for unknown new classes. For all incremental new sessions, we freeze the backbone network and employ prototype rectification without further training to refine the prototypes of the novel classes. We conduct extensive experiments with different input scales, including federated cross-domain pretraining and cross-domain class-incremental experiments. BFCP efficiently handles both novel and base classes of each incremental session and significantly outperforms state-of-the-art methods, achieving an average accuracy of 63.47% on the CIFAR100 dataset.

Few-shot learning (FSL) is challenging due to the scarce labeled novel-class data. Researchers have to train the embedding function with auxiliary base-class data to obtain the novel-class embeddings. However, the domain gap makes the novel-class embedding unsatisfactory, as the novel class and the base class are disjoint. Recent studies prove that embedding rectification shows great potential, introduces miscellaneous variants, and achieves similar performances. Nonetheless, while each method demonstrates unique strengths, they often address distinct challenges in isolation, limiting their applicability in more complex or diverse scenarios. In this article, we take a closer look at these methods and hypothesize that a general embedding rectification framework is more essential to the model's performance. To verify our observation, we propose: 1) a distribution propagation (DisP) layer distinguishes the inter-class margin and increases intra-class aggregation, performing the task-level rectification; and 2) a prototype propagation (ProtoP) layer moves the prototype toward the ideal class center, applying the prototype-query level rectification. Our framework aims to maximize the actual data distribution. Although pseudo-labeling proves effective in achieving this goal, a significant challenge is ensuring the reliable retention of only high-confidence predictions. To overcome this, we introduce a distribution-based pseudo-labeling method pseudo-query upgrade (PseQUp) that provides more reliable pseudo-labeling samples without relying on confidence scores. We evaluate the proposed method in both transfer learning and meta-learning scenarios. Empirical experiments show the applicable and plug-and-play ability of the proposed methods.

Few-shot image generation aims to generate novel images for an unseen category with only a few samples. Prior studies fail to produce novel images with desirable diversity and fidelity. To ameliorate the generation quality, we in this paper propose a Semantic-Aware Generator (SAG) to provide explicit semantic guidance to the discriminator, and a Low-level Feature Augmentation (LFA) technique to provide fine-grained information, facilitating the diversity. Specifically, we observe that the generator feature layers contain different levels of semantic information. Such observation motivates us to employ intermediate feature maps of the generator as semantic labels to guide the discriminator, improving the semantic awareness of the generator. Moreover, spatially informative and diverse features obtained via LFA contribute to better generation quality. Together with the aforementioned module, we conduct extensive experiments on three representative benchmarks and the results demonstrate the effectiveness and advancement of our method.

In the face of high-dimensional and complex data, effective subspace can preserve specific statistical properties and provide an appropriate representation of data, which generally facilitates the underlying tasks such as clustering or classification. Meanwhile, multiple kernel learning is a technique to combine multiple kernels from different feature spaces effectively. Thus, by incorporating multiple kernels into the process of subspace learning, different feature spaces can be projected into a unified subspace. This paper proposes the Multiple Kernel Subspace Learning (MKSL) for embedding the original space into a unified subspace. Multiple kernels of different feature spaces are combined by MKSL in the process of learning, which can extend the suitability for various applications. Moreover, to generate the optimal combination kernel of subspace learning, we propose a two-step iteration strategy to learn the appropriate kernel weights and transformation matrix of projecting simultaneously. Furthermore, our proposed formulation of MKSL can introduce different prior knowledge such as class information and neighborhood relationships. Thus it is competent to the unsupervised learning, semi-supervised learning, and supervised learning. Extensive experiments are conducted on diverse datasets, and the performances are comprehensively evaluated on different tasks. The experimental results indicate that the proposed algorithm is outstanding in unsupervised clustering task and effective in supervised and semi-supervised classification tasks.

In few-shot learning, methods are enslaved to the scarce labeled data, resulting in suboptimal embedding. Recent studies learn the embedding network by other large-scale labeled data. However, the trained network may give rise to the distorted embedding of target data. We argue two respects are required for an unprecedented and promising solution. We call them Better Embedding and More Shots (BEMS). Suppose we propose to extract embedding from the embedding network. BE maximizes the extraction of general representation and prevents over-fitting information. For this purpose, we introduce the topological relation for global reconstruction, avoiding excessive memorizing. MS maximizes the relevance between the reconstructed embedding and the target class space. In this respect, increasing the number of shots is a pivotal but intractable strategy. As a creative method, we derive the bound of information-theory-based loss function and implicitly achieve infinite shots with negligible cost. A substantial experimental analysis is carried out to demonstrate the state-of-the-art performance. Compared to the baseline, our method improves by up to 10%+. We also prove that BEMS is suitable for both standard pre-trained and meta-learning embedded networks.

Training GANs under limited data often leads to discriminator overfitting and memorization issues, causing divergent training. Existing approaches mitigate the overfitting by employing data augmentations, model regularization, or attention mechanisms. However, they ignore the frequency bias of GANs and take poor consideration towards frequency information, especially high-frequency signals that contain rich details. To fully utilize the frequency information of limited data, this paper proposes FreGAN, which raises the model's frequency awareness and draws more attention to synthesising high-frequency signals, facilitating high-quality generation. In addition to exploiting both real and generated images' frequency information, we also involve the frequency signals of real images as a self-supervised constraint, which alleviates the GAN disequilibrium and encourages the generator to synthesis adequate rather than arbitrary frequency signals. Extensive results demonstrate the superiority and effectiveness of our FreGAN in ameliorating generation quality in the low-data regime (especially when training data is less than 100). Besides, FreGAN can be seamlessly applied to existing regularization and attention mechanism models to further boost the performance.

In the Few-Shot Learning (FSL), much of the related efforts only rely on the few available labeled samples (support set) building approach. However, the challenge is that the support set is easy-to-be-biased, so that they cannot be competent prototypes and are hard to represent the class distribution, leading to performance bottlenecks. In this paper, we propose to solve this obstacle by capturing the distribution of the unlabeled samples (query set). We propose two sampling methods: DeepSearch ($\cal DS$) and WideSearch ($\cal WS$). Both approaches are simple to implement and have no trainable parameters. They search the query samples near to the support set in different manners. Afterward, the statistic information is calculated, and we generate the latent samples according to it. The generated latent set is promising. First, it brings the query set distribution information to the classifier, which significantly improves the performance of the cross-entropy-based classifier. Second, it helps the support set become the better prototypes, which boosts the performance of the prototype-based classifier. Third, we find few latent samples are enough to boost the performance. Abundant experiments prove the proposed method achieves state-of-the-art performance on the few-shot tasks. Finally, rich ablation studies explain the compelling details of our approach.

Multi-instance learning (MIL) has garnered considerable attention in recent years due to its favorable performance in various scenarios. Nonetheless, most previous studies have implicitly expressed the correlation between instances and bags. Moreover, the importance of negative instances has been largely overlooked. Hence, we seek to present an explicit and intuitively understandable method that can compensate for these deficiencies. In this article, we creatively introduce a metric-based multiconcept MIL approach based on two aspects. First, the triplet-based bag embedding method identifies instance categories and builds attention weights for every instance explicitly. Accordingly, bag embedding is accomplished under the limitation of weak supervision. Second, the developed instance correlation metric approach in the superbag considers the multiconcept issue to boost the model generalization performance. We have designed a rich variety of experiments to demonstrate the performance of our algorithm. The artificial data experiment reveals the interpretability of the proposed network. The results of the comparison experiment confirm that our method shows favorable performance in multiple tasks. Finally, we illustrate the motivation of the presented method by the ablation experiments.

Sampling method is one of the most commonly used techniques in dealing with imbalanced data. Most of the existing undersampling methods randomly select samples from negative class with replacement. However, it may lose some important information of the training data. Moreover, increasing the positive data by oversampling in high imbalanced situations may cause the overlapping problem. To overcome these problems, this paper proposes a hybrid sampling method. The method takes the distributions of the training data into consideration by the information entropy, thus distinguishing the important samples in the undersampling procedure. Meanwhile, since the positive data only extend to the size of each subset of the negative class in the oversampling, the overlapping problem is relieved. Further, the method retains all the data in the training procedure and generates various data views from the original training data. Then each view is handled with an individual basic classifier. Finally, all the basic classifiers are combined by the ensemble method. The newly proposed method is named as Entropy-based Hybrid Sampling Ensemble Learning (EHSEL). In addition, the EHSEL is applied to three different kinds of basic classifiers to validate its robustness. Experiments results show the great effectiveness of the EHSEL on real-world imbalanced data sets.