Few-shot learning seeks to emulate humans by grasping a new concept with a few examples. However, it is often a tricky problem to completely learn a new concept and avoid falling into overfitting with few examples. Recently, self-supervision has been introduced as an auxiliary pretext task to enhance the generalization of few-shot learning models. However, in few-shot scenarios, self-supervised learning may easily learn undesirable shortcuts and biased representations due to insufficient information, negatively impacting the decision boundary of few-shot models. In this paper, we propose a new learning paradigm called conditional self-supervised learning (CSS), where the blind self-supervised tasks are constrained. Specifically, we first utilize the embedding knowledge of supervised learning as a condition to guide the representation learning process of self-supervised tasks. In order to optimize the overall objectives, we formulate the CSS as a multiple objective optimization problem to find Pareto optimal solutions. Moreover, we combine the meaningful information extracted from both supervised and self-supervised learning into a unified distribution, further enriching the original representation. Extensive experiments show that our method, without any fine-tuning, significantly improves accuracy in few-shot tasks compared to state-of-the-art methods.

Human intelligence is characterized by our ability to absorb and apply knowledge from the world around us, especially in rapidly acquiring new concepts from minimal examples, underpinned by prior knowledge. Few-shot learning (FSL) aims to mimic this capacity by enabling significant generalizations and transferability. However, traditional FSL frameworks often rely on assumptions of clean, complete, and static data, conditions that are seldom met in real-world environments. Such assumptions falter in the inherently uncertain, incomplete, and dynamic contexts of the open world. This paper presents a comprehensive review of recent advancements designed to adapt FSL to open-world environments. We categorize existing methods into three distinct types of FSL in the open world: those involving varying instances, varying classes, and varying distributions. Each category is discussed in terms of its specific challenges and methods, as well as its strengths and weaknesses. We standardize experimental settings and metric benchmarks across scenarios and provide a comparative analysis of the performance of various methods. In conclusion, we outline potential future research directions for this evolving field. It is our hope that this review will catalyze further development of effective solutions to these complex challenges, thereby advancing the field of artificial intelligence.

Owing to the excellent capability in dealing with label ambiguity, Label Distribution Learning (LDL), as an emerging machine learning paradigm, has received extensive research in recent years. Though remarkable progress has been achieved in various tasks, one limitation with existing LDL methods is that they are all based on the i.i.d. assumption that training and test data are identically and independently distributed. As a result, they suffer obvious performance degradation and are no longer applicable when tested in out-of-distribution scenarios, which severely limits the application of LDL in many tasks. In this paper, we identify and investigate the Generalizable Label Distribution Learning (GLDL) problem. To handle such a challenging problem, we delve into the characteristics of GLDL and find that the label annotations changing with the variability of the domains is the underlying reason for the performance degradation of the existing methods. Inspired by this observation, we explore domain-invariant feature-label correlation information to reduce the impact of label annotations changing with domains and propose two practical methods. Extensive experiments verify the superior performance of the proposed methods. Our work fills the gap in benchmarks and techniques for practical GLDL problems.

Recently, noisy few-shot learning (NFSL) has been exploring the model robustness to label noise, breaking the limitation of completely accurate labeling in small sample scenarios. Existing NFSL methods directly employ delicately designed pseudo-noise to simulate and adapt to noisy environments. However, determining the optimal combination of pseudo-noise is challenging and improperly configuring pseudo-noise may lead to adverse effects on the training models. To deal with the problems, this letter proposes a novelAdaptiveMultI-viewDenoisingEvaluation (AMIDE) framework, which establishes an adaptive and robust embedding and classifier without relying on pseudo-noise. In the training phase, we design an adaptive label smoothing scheme, where soft labels with learnable smooth coefficients are inferred from data distribution to mitigate overconfident labeling. In the testing stage, we propose a multi-view fused evaluation scheme, where different network layers are treated as distinct views to generate potential clean features and modify prototypes, thereby enhancing the accuracy of evaluation. In this way, the impact of noise is effectively alleviated from two perspectives. Extensive experiments on several few-shot classification benchmarks show the superiority and robustness of our method.

Multi-Label Learning (MLL) involves the task of assigning a set of relevant labels to a given instance. Recently, Label Enhancement (LE) has gained significant attention in various MLL tasks, as it allows for effective mining the implicit relative importance information of different labels. However, in existing LE-based MLL methods, the LE process is decoupled from the MLL process. Consequently, the label distribution recovered by the LE process may not be suitable for training the predictive model, thus affecting the overall learning system. In this study, we propose a novel approach named interactive Fusion Label Enhancement for Multi-Label Learning ( Flem ) that seamlessly integrates the LE process with the MLL process. Specifically, we introduce a matching and interaction mechanism comprising a novel interaction label enhancement loss and a contrastive alignment approach to prevent object mismatch. Furthermore, we present a unified label distribution loss that establishes the relationship between the recovered label distribution and the training of the predictive model. By leveraging these losses, the label distributions obtained from the LE process can be efficiently utilized for training the predictive model. Experimental results on multiple benchmark datasets demonstrate the effectiveness of the proposed method.

Multi-label few-shot learning (ML-FSL) refers to the task of tagging previously unseen images with a set of relevant labels, giving a small number of training examples. Modeling the correlations between instances and labels, formulated in the existing methods, allows us to extract more available knowledge from limited examples. However, they simply explore the instance and label correlations with a uniform importance assumption without considering the discrepancy of importance in different instances or labels, making the utilization of instance and label correlations a bottleneck for ML-FSL. To tackle the issue, we propose a unified framework named bilateral correlation reconstruction (BCR) to enable the network to effectively mine underlying instance and label correlations with varying importance information from both instance-to-label and label-to-instance perspectives. Specifically, from the instance-to-label perspective, we refine prototypes per category by reweighting each image with its specific instance-importance degree extracted from the similarity between the instance and the corresponding category. From the label-to-instance perspective, we smooth labels for each image by recovering latent label-importance with considering the integrated topology of all samples in a task. Experimental results on multiple benchmarks validate that BCR could outperform existing ML-FSL methods by large margins.

Multi-label classification (MLC) refers to the problem of tagging a given instance with a set of relevant labels. Most existing MLC methods are based on the assumption that the correlation of two labels in each label pair is symmetric, which is violated in many real-world scenarios. Moreover, most existing methods design learning processes associated with the number of labels, which makes their computational complexity a bottleneck when scaling up to large-scale output space. To tackle these issues, we propose a novel method named scalable label distribution learning (SLDL) for MLC, which can describe different labels as distributions in a latent space, where the label correlation is asymmetric and the dimension is independent of the number of labels. Specifically, SLDL first converts labels into continuous distributions within a low-dimensional latent space and leverages the asymmetric metric to establish the correlation between different labels. Then, it learns the mapping from the feature space to the latent space, resulting in the computational complexity is no longer related to the number of labels. Finally, SLDL leverages a nearest neighbor-based strategy to decode the latent representations and obtain the final predictions. Extensive experiments illustrate that SLDL achieves very competitive classification performances with little computational consumption.

Few-shot learning is an open problem to learning a new concept with little supervision from limited labeled data. As an alternative knowledge for few-shot learning, self-supervised learning can extract supervisory signals directly from unlabeled data. However, existing self-supervised few-shot methods which directly take the summation of two tasks, have two fundamental bottlenecks: 1) representation bias: how to extract efficacious supervisory signals in self-supervision and eliminate the disturbance of undesirable shortcuts with limited examples, 2) objective conflict: how to adaptively trade-off the self-supervision and supervision to achieve the optimal model performance. To address the above problems, in this paper, we propose a novel approach named ReInforced SElf-supervised training (RISE) for few-shot learning. RISE leverages agent-relative supervision to eliminate the undesirable shortcut learning of self-supervised training. Meanwhile, it dynamically explores the balance between supervisory signals from self-supervised tasks and inherent supervision from few-shot tasks to avoid the trade-off dilemma. Therefore, the new pattern for training self-supervision can be resilient to few-shot learning and enhance the performance for few-shot identification. Extensive experiments on several public benchmark datasets verify the effectiveness of our approach.

Multi-label text classification (MLTC) refers to the problem of tagging a given document with the most relevant subset of labels. One of the biggest challenges for MLTC is the existence of class imbalance. Most advanced MLTC models suffer from this issue, which limits the performance of the models. In this paper, we propose a model-agnostic framework named variational continuous label distribution learning (VCLDL) to address this problem. VCLDL theoretically builds a corresponding relationship between the feature space and the label space to mine the information hidden in the observable logical labels. Specifically, VCLDL regards label distribution as a continuous density function in latent space and forms a flexible variational approach to approximate the density function of the labels with the collaboration of the feature space. Combined with VCLDL, MLTC models can pay more attention to the distribution of the whole label set, rather than specific labels with maximum response values, thus the class imbalance problem can be well overcome. Experimental results on multiple benchmark datasets demonstrate that VCLDL can bring significant performance improvements over the existing MLTC models.

Label distribution covers a certain number of labels, representing the degree to which each label describes an instance. The learning process on the instances labeled by label distributions is called Label Distribution Learning (LDL). Although LDL has been applied successfully to many practical applications, one problem with existing LDL methods is that they are limited to data with balanced label information. However, annotation information in real-world data often exhibits imbalanced distributions, which significantly degrades the performance of existing methods. In this paper, we investigate the Imbalanced Label Distribution Learning (ILDL) problem. To handle this challenging problem, we delve into the characteristics of ILDL and empirically find that the representation distribution shift is the underlying reason for the performance degradation of existing methods. Inspired by this finding, we present a novel method named Representation Distribution Alignment (RDA). RDA aligns the distributions of feature representations and label representations to alleviate the impact of the distribution gap between the training set and the test set caused by the imbalance issue. Extensive experiments verify the superior performance of RDA. Our work fills the gap in benchmarks and techniques for practical ILDL problems.

Few-shot learning which aims to generalize knowledge learned from annotated base training data to recognize unseen novel classes has attracted considerable attention. Existing few-shot methods rely on completely clean training data. However, in the real world, the training data are always corrupted and accompanied by noise due to the disturbance in data transmission and low-quality annotation, which severely degrades the performance and generalization capability of few-shot models. To address the problem, we propose a unified peer-collaboration learning (PCL) framework to extract valid knowledge from corrupted data for few-shot learning. PCL leverages two modules to mimic the peer collaboration process which cooperatively evaluates the importance of each sample. Specifically, each module first estimates the importance weights of different samples by encoding the information provided by the other module from both global and local perspectives. Then, both modules leverage the obtained importance weights to guide the reevaluation of the loss value of each sample. In this way, the peers can mutually absorb knowledge to improve the robustness of few-shot models. Experiments verify that our framework combined with different few-shot methods can significantly improve the performance and robustness of original models.

Few-shot open-set recognition (FSOSR) has become a great challenge, which requires classifying known classes and rejecting the unknown ones with only limited samples. Existing FSOSR methods mainly construct an ambiguous distribution of known classes from scarce known samples without considering the latent distribution information of unknowns, which degrades the performance of open-set recognition. To address this issue, we propose a novel loss function called multi-relation margin (MRM) loss that can plug in few-shot methods to boost the performance of FSOSR. MRM enlarges the margin between different classes by extracting the multi-relationship of paired samples to dynamically refine the decision boundary for known classes and implicitly delineate the distribution of unknowns. Specifically, MRM separates the classes by enforcing a margin while concentrating samples of the same class on a hypersphere with a learnable radius. In order to better capture the distribution information of each class, MRM extracts the similarity and correlations among paired samples, ameliorating the optimization of the margin and radius. Experiments on public benchmarks reveal that methods with MRM loss can improve the unknown detection of AUROC by a significant margin while correctly classifying the known classes.

Label Distributed Learning (LDL) is an emerging machine learning paradigm that has received extensive research in recent years. Owing to the excellent capability in dealing with label ambiguity, LDL has been widely adopted in many real-world scenarios. Though remarkable progress has been achieved in various tasks, one limitation with existing LDL methods is that they are all based on the i.i.d. assumption that training and test data are identically and independently distributed. As a result, they suffer obvious performance degradation and are no longer applicable when tested in out-of-distribution scenarios, which severely limits the application of LDL in many tasks. In this paper, we identify and investigate the Generalizable Label Distribution Learning (GLDL) problem. To handle such a challenging problem, we delve into the characteristics of GLDL and find that feature-label correlation and label-label correlation are two essential subjects in GLDL. Inspired by this finding, we propose a simple yet effective model-agnostic framework named Domain-Invariant Correlation lEarning (DICE). DICE mines and utilizes the correlation between feature and label that are invariant across different domains to learn a generalizable feature-label correlation by introducing a prior alignment strategy. In the meantime, it leverages a label correlation alignment strategy to further retain the consistency of label-label correlation in different domains. Extensive experiments verify the superior performance of DICE. Our work fills the gap in benchmarks and techniques for practical GLDL problems.

Robust few-shot learning (RFSL), which aims to address noisy labels in few-shot learning, has recently gained considerable attention. Existing RFSL methods are based on the assumption that the noise comes from known classes (in-domain), which is inconsistent with many real-world scenarios where the noise does not belong to any known classes (out-of-domain). We refer to this more complex scenario as open-world few-shot learning (OFSL), where in-domain and out-of-domain noise simultaneously exists in few-shot datasets. To address the challenging problem, we propose a unified framework to implement comprehensive calibration from instance to metric. Specifically, we design a dual-networks structure composed of a contrastive network and a meta network to respectively extract feature-related intra-class information and enlarged inter-class variations. For instance-wise calibration, we present a novel prototype modification strategy to aggregate prototypes with intra-class and inter-class instance reweighting. For metric-wise calibration, we present a novel metric to implicitly scale the per-class prediction by fusing two spatial metrics respectively constructed by the two networks. In this way, the impact of noise in OFSL can be effectively mitigated from both feature space and label space. Extensive experiments on various OFSL settings demonstrate the robustness and superiority of our method. Our source codes is available at https://github.com/anyuexuan/IDEAL.

Multi-label learning (MLL) refers to the problem of tagging a given instance with a set of relevant labels. In MLL, the implicit relative importance of different labels representing a single instance is generally different, which recently gained considerable attention and should be fully leveraged. Therefore, label enhancement (LE) has been widely applied in various MLL tasks as the ability to effectively mine the implicit relative importance information of different labels. However, due to the fact that the label enhancement process in previous LE-based MLL methods is decoupled from the training process on the predictive models, the objective of LE does not match the training process and finally affects the whole learning system. In this paper, we propose a novel approach named Fusion Label Enhancement for Multi-label learning (FLEM) to effectively integrate the LE process and the training process. Specifically, we design a matching and interaction mechanism which leverages a novel interaction label enhancement loss to avoid that the recovered label distribution does not match the need of the predictive model. In the meantime, we present a unified label distribution loss for establishing the corresponding relationship between the recovered label distribution and the training of the predictive model. With the proposed loss, the label distributions recovered from the LE process can be efficiently utilized for training the predictive model. Experimental results on multiple benchmark datasets validate the effectiveness of the proposed approach.

How to learn a transferable feature representation from limited examples is a key challenge for few-shot classification. Self-supervision as an auxiliary task to the main supervised few-shot task is considered to be a conceivable way to solve the problem since self-supervision can provide additional structural information easily ignored by the main task. However, learning a good representation by traditional self-supervised methods is usually dependent on large training samples. In few-shot scenarios, due to the lack of sufficient samples, these self-supervised methods might learn a biased representation, which more likely leads to the wrong guidance for the main tasks and finally causes the performance degradation. In this paper, we propose conditional self-supervised learning (CSS) to use auxiliary information to guide the representation learning of self-supervised tasks. Specifically, CSS leverages supervised information as prior knowledge to shape and improve the learning feature manifold of self-supervision without auxiliary unlabeled data, so as to reduce representation bias and mine more effective semantic information. Moreover, CSS exploits more meaningful information through supervised and the improved self-supervised learning respectively and integrates the information into a unified distribution, which can further enrich and broaden the original representation. Extensive experiments demonstrate that our proposed method without any fine-tuning can achieve a significant accuracy improvement on the few-shot classification scenarios compared to the state-of-the-art few-shot learning methods.