Recent advances in weakly multi-label learning (MLL) have demonstrated impressive potential in multi-label classification tasks. Unfortunately, collecting multi-labels for each instance proves to be time-consuming and labor-intensive, since these MLL methods requires the assessment of all the candidate labels. To alleviate this challenge, a novel labeling setting termed Determined Multi-Label Learning is proposed, aiming to effectively reduce the cost for browsing labels in multi-label tasks. In this setting, each training instance is associated with a determined multi-label, which indicates whether the instance contains the provided class label. Besides, each instance only need to be determined once, which significantly reduce the annotation cost of the labeling task for multi-label datasets. In this paper, we theoretically derive an risk-consistent estimator to learn from these determined-labeled training data. Additionally, we introduce a similarity-based prompt learning method, which minimizes the risk-consistent loss of large-scale pre-trained models to learn a supplemental prompt with richer semantic information. Extensive experimental validation underscores the efficacy of our approach. Our code is available at the link: https://github.com/WilsonMqz/DMLL

To reduce pressure of manual annotation, researchers have explored various weakly supervised learning methods and achieved remarkable results in multi-class classification tasks. However, these methods still require annotating from the entire set of candidate labels, which becomes particularly time-consuming when the labeling space is large. To alleviate this problem, we propose a novel labeling mechanism called stochastic labels, which reduces the time spent browsing labeling space by annotating the instance from a small labels subset. In this paper, we introduce an unbiased risk estimator and establish a prototype baseline to learn a multi-class classifier from these stochastic labels. Besides, we derive the estimation error bound of the proposed method, showing that the empirical risk could converge to the true classification risk as the number of training samples increases. Finally, we conduct extensive experiments on widely-used benchmark datasets to validate the effectiveness of our approach. Our method surpasses state-of-the-art weakly supervised methods, highlighting its efficiency and robustness. Our code is available at: https://github.com/WilsonMqz/SLL

Pre-trained Vision-Language Models (VLMs) exhibit strong zero-shot classification abilities, demonstrating great potential for generating weakly supervised labels. Unfortunately, existing weakly supervised learning methods are short of ability in generating accurate labels via VLMs. In this paper, we propose a novel weakly supervised labeling setting, namely True-False Labels (TFLs) which can achieve high accuracy when generated by VLMs. The TFL indicates whether an instance belongs to the label, which is randomly and uniformly sampled from the candidate label set. Specifically, we theoretically derive a risk-consistent estimator to explore and utilize the conditional probability distribution information of TFLs. Besides, we propose a convolutional-based Multi-modal Prompt Retrieving (MRP) method to bridge the gap between the knowledge of VLMs and target learning tasks. Experimental results demonstrate the effectiveness of the proposed TFL setting and MRP learning method. The code to reproduce the experiments is at https://github.com/Tranquilxu/TMP.

Recent advances in vision-language models (VLMs) have demonstrated remarkable capabilities in image classification by leveraging predefined sets of labels to construct text prompts for zero-shot reasoning. However, these approaches face significant limitations in undefined domains, where the label space is vocabulary-unknown and composite. We thus introduce Generative Semantic Labels (GSLs), a novel task that aims to predict a comprehensive set of semantic labels for an image without being constrained by a predefined labels set. Unlike traditional zero-shot classification, GSLs generates multiple semantic-level labels, encompassing objects, scenes, attributes, and relationships, thereby providing a richer and more accurate representation of image content. In this paper, we propose Chain-of-Action (CoA), an innovative method designed to tackle the GSLs task. CoA is motivated by the observation that enriched contextual information significantly improves generative performance during inference. Specifically, CoA decomposes the GSLs task into a sequence of detailed actions. Each action extracts and merges key information from the previous step, passing enriched context to the next, ultimately guiding the VLM to generate comprehensive and accurate semantic labels. We evaluate the effectiveness of CoA through extensive experiments on widely-used benchmark datasets. The results demonstrate significant improvements across key performance metrics, validating the capability of CoA to generate accurate and contextually rich semantic labels. Our work not only advances the state-of-the-art in generative semantic labels but also opens new avenues for applying VLMs in open-ended and dynamic real-world scenarios.

Learning from Multi-Positive and Unlabeled (MPU) data has gradually attracted significant attention from practical applications. Unfortunately, the risk of MPU also suffer from the shift of minimum risk, particularly when the models are very flexible. In this paper, to alleviate the shifting of minimum risk problem, we propose an Example Sieve Approach (ESA) to select examples for training a multi-class classifier. Specifically, we sieve out some examples by utilizing the Certain Loss (CL) value of each example in the training stage and analyze the consistency of the proposed risk estimator. Besides, we show that the estimation error of proposed ESA obtains the optimal parametric convergence rate. Extensive experiments on various real-world datasets show the proposed approach outperforms previous methods.

Multi-label learning (MLL) learns from samples associated with multiple labels, where it is expensive and time consuming to provide detailed annotation for each sample in real-world datasets. To deal with this challenge, single positive multi-label learning (SPML) has been studied in recent years. In SPML, each sample is annotated with only one positive label, which is much easier and less costly. However, in many real-world scenarios, single positive labels may have global unannotated categories (GUCs) in annotation process, which exist in the label space but do not serve as single positive label for any samples. Unfortunately, previous SPML approaches are less applicable to classify GUCs due to the absence of supervised information. To solve this problem, we propose a novel prompt expanding framework that leverages a large-scale pretrained vision and language model called the Recognize Anything Model (RAM) to offer supervision signals for GUCs. Specifically, we first provide a simple but effective strategy to generate reliable pseudo-labels for GUCs by utilizing zero-shot predictions of RAM. Subsequently, we introduce additional prompts from a large common category list and fuse them by learnable weighting factors, which expends the semantic representation of GUCs. Experiments show that our method achieves state-of-the-art results on all four benchmarks. The code to reproduce the experiments is at: https://github.com/yingpenga/VLSPE

Long-tailed data is prevalent in real-world classification tasks and heavily relies on supervised information, which makes the annotation process exceptionally labor-intensive and time-consuming. Unfortunately, despite being a common approach to mitigate labeling costs, existing weakly supervised learning methods struggle to adequately preserve supervised information for tail samples, resulting in a decline in accuracy for the tail classes. To alleviate this problem, we introduce a novel weakly supervised labeling setting called Reduced Label. The proposed labeling setting not only avoids the decline of supervised information for the tail samples, but also decreases the labeling costs associated with long-tailed data. Additionally, we propose an straightforward and highly efficient unbiased framework with strong theoretical guarantees to learn from these Reduced Labels. Extensive experiments conducted on benchmark datasets including ImageNet validate the effectiveness of our approach, surpassing the performance of state-of-the-art weakly supervised methods. Source code is available at \hrefhttps://github.com/WilsonMqz/LTRL https://github.com/WilsonMqz/LTRL

Annotating data for sensitive labels (e.g., disease, smoking) poses a potential threats to individual privacy in many real-world scenarios. To cope with this problem, we propose a novel setting to protect privacy of each instance, namely learning from concealed labels for multi-class classification. Concealed labels prevent sensitive labels from appearing in the label set during the label collection stage, which specifies none and some random sampled insensitive labels as concealed labels set to annotate sensitive data. In this paper, an unbiased estimator can be established from concealed data under mild assumptions, and the learned multi-class classifier can not only classify the instance from insensitive labels accurately but also recognize the instance from the sensitive labels. Moreover, we bound the estimation error and show that the multi-class classifier achieves the optimal parametric convergence rate. Experiments demonstrate the significance and effectiveness of the proposed method for concealed labels in synthetic and real-world datasets. Source code is available at https://github.com/WilsonMqz/CLF