Vision-Language Models (VLMs), with their powerful content generation capabilities, have been successfully applied to data annotation processes. However, the VLM-generated labels exhibit dual limitations: low quality (i.e., label noise) and absence of error correction mechanisms. To enhance label quality, we propose Human-Corrected Labels (HCLs), a novel setting that efficient human correction for VLM-generated noisy labels. As shown in Figure 1(b), HCL strategically deploys human correction only for instances with VLM discrepancies, achieving both higher-quality annotations and reduced labor costs. Specifically, we theoretically derive a risk-consistent estimator that incorporates both human-corrected labels and VLM predictions to train classifiers. Besides, we further propose a conditional probability method to estimate the label distribution using a combination of VLM outputs and model predictions. Extensive experiments demonstrate that our approach achieves superior classification performance and is robust to label noise, validating the effectiveness of HCL in practical weak supervision scenarios.

Recent advances in weakly multi-label learning (MLL) have demonstrated impressive potential in multi-label classification tasks. Unfortunately, collecting multi-labels for each instance proves to be time-consuming and labor-intensive, since these MLL methods requires the assessment of all the candidate labels. To alleviate this challenge, a novel labeling setting termed Determined Multi-Label Learning is proposed, aiming to effectively reduce the cost for browsing labels in multi-label tasks. In this setting, each training instance is associated with a determined multi-label, which indicates whether the instance contains the provided class label. Besides, each instance only need to be determined once, which significantly reduce the annotation cost of the labeling task for multi-label datasets. In this paper, we theoretically derive an risk-consistent estimator to learn from these determined-labeled training data. Additionally, we introduce a similarity-based prompt learning method, which minimizes the risk-consistent loss of large-scale pre-trained models to learn a supplemental prompt with richer semantic information. Extensive experimental validation underscores the efficacy of our approach. Our code is available at the link: https://github.com/WilsonMqz/DMLL

To reduce pressure of manual annotation, researchers have explored various weakly supervised learning methods and achieved remarkable results in multi-class classification tasks. However, these methods still require annotating from the entire set of candidate labels, which becomes particularly time-consuming when the labeling space is large. To alleviate this problem, we propose a novel labeling mechanism called stochastic labels, which reduces the time spent browsing labeling space by annotating the instance from a small labels subset. In this paper, we introduce an unbiased risk estimator and establish a prototype baseline to learn a multi-class classifier from these stochastic labels. Besides, we derive the estimation error bound of the proposed method, showing that the empirical risk could converge to the true classification risk as the number of training samples increases. Finally, we conduct extensive experiments on widely-used benchmark datasets to validate the effectiveness of our approach. Our method surpasses state-of-the-art weakly supervised methods, highlighting its efficiency and robustness. Our code is available at: https://github.com/WilsonMqz/SLL

Pre-trained Vision-Language Models (VLMs) exhibit strong zero-shot classification abilities, demonstrating great potential for generating weakly supervised labels. Unfortunately, existing weakly supervised learning methods are short of ability in generating accurate labels via VLMs. In this paper, we propose a novel weakly supervised labeling setting, namely True-False Labels (TFLs) which can achieve high accuracy when generated by VLMs. The TFL indicates whether an instance belongs to the label, which is randomly and uniformly sampled from the candidate label set. Specifically, we theoretically derive a risk-consistent estimator to explore and utilize the conditional probability distribution information of TFLs. Besides, we propose a convolutional-based Multi-modal Prompt Retrieving (MRP) method to bridge the gap between the knowledge of VLMs and target learning tasks. Experimental results demonstrate the effectiveness of the proposed TFL setting and MRP learning method. The code to reproduce the experiments is at https://github.com/Tranquilxu/TMP.

Recent advances in vision-language models (VLMs) have demonstrated remarkable capabilities in image classification by leveraging predefined sets of labels to construct text prompts for zero-shot reasoning. However, these approaches face significant limitations in undefined domains, where the label space is vocabulary-unknown and composite. We thus introduce Generative Semantic Labels (GSLs), a novel task that aims to predict a comprehensive set of semantic labels for an image without being constrained by a predefined labels set. Unlike traditional zero-shot classification, GSLs generates multiple semantic-level labels, encompassing objects, scenes, attributes, and relationships, thereby providing a richer and more accurate representation of image content. In this paper, we propose Chain-of-Action (CoA), an innovative method designed to tackle the GSLs task. CoA is motivated by the observation that enriched contextual information significantly improves generative performance during inference. Specifically, CoA decomposes the GSLs task into a sequence of detailed actions. Each action extracts and merges key information from the previous step, passing enriched context to the next, ultimately guiding the VLM to generate comprehensive and accurate semantic labels. We evaluate the effectiveness of CoA through extensive experiments on widely-used benchmark datasets. The results demonstrate significant improvements across key performance metrics, validating the capability of CoA to generate accurate and contextually rich semantic labels. Our work not only advances the state-of-the-art in generative semantic labels but also opens new avenues for applying VLMs in open-ended and dynamic real-world scenarios.

Learning from Multi-Positive and Unlabeled (MPU) data has gradually attracted significant attention from practical applications. Unfortunately, the risk of MPU also suffer from the shift of minimum risk, particularly when the models are very flexible. In this paper, to alleviate the shifting of minimum risk problem, we propose an Example Sieve Approach (ESA) to select examples for training a multi-class classifier. Specifically, we sieve out some examples by utilizing the Certain Loss (CL) value of each example in the training stage and analyze the consistency of the proposed risk estimator. Besides, we show that the estimation error of proposed ESA obtains the optimal parametric convergence rate. Extensive experiments on various real-world datasets show the proposed approach outperforms previous methods.

Cross-modal remote sensing image-text retrieval (CMRSITR) involves retrieving relevant samples in one modality based on a query from another modality. Previous dense retrieval methods utilizing multivector dense representations have significantly enhanced retrieval performance. Meanwhile, recent advances in sparse retrieval have demonstrated that sparse representations offer comparable performance with enhanced interpretability and faster retrieval speeds. However, effectively integrating the strengths of these two paradigms to enable efficient and accurate retrieval in large-scale remote sensing (RS) image-text datasets remains an open challenge. In this study, we propose sparse-guided partial dense (SGPD) cross-modal retrieval, a novel approach that efficiently transforms dense vectors from pretrained dense retrieval models into sparse representations and leverages the overlap between sparse retrieval results and dense vector clusters to achieve high-precision and fast retrieval. By probabilistically selecting a limited number of dense clusters containing top sparse results, SGPD ensures retrieval efficiency while minimizing both memory and time costs. Designed as a plug-and-play solution, SGPD can be seamlessly integrated into existing RS image-text retrieval (RSITR) models without requiring modifications to their architectures. Extensive experiments on RS image-text datasets of varying scales demonstrate that SGPD achieves retrieval accuracy comparable to dense retrieval methods while significantly reducing training time and memory consumption.

Multi-label learning (MLL) learns from samples associated with multiple labels, where it is expensive and time consuming to provide detailed annotation for each sample in real-world datasets. To deal with this challenge, single positive multi-label learning (SPML) has been studied in recent years. In SPML, each sample is annotated with only one positive label, which is much easier and less costly. However, in many real-world scenarios, single positive labels may have global unannotated categories (GUCs) in annotation process, which exist in the label space but do not serve as single positive label for any samples. Unfortunately, previous SPML approaches are less applicable to classify GUCs due to the absence of supervised information. To solve this problem, we propose a novel prompt expanding framework that leverages a large-scale pretrained vision and language model called the Recognize Anything Model (RAM) to offer supervision signals for GUCs. Specifically, we first provide a simple but effective strategy to generate reliable pseudo-labels for GUCs by utilizing zero-shot predictions of RAM. Subsequently, we introduce additional prompts from a large common category list and fuse them by learnable weighting factors, which expends the semantic representation of GUCs. Experiments show that our method achieves state-of-the-art results on all four benchmarks. The code to reproduce the experiments is at: https://github.com/yingpenga/VLSPE

Long-tailed data is prevalent in real-world classification tasks and heavily relies on supervised information, which makes the annotation process exceptionally labor-intensive and time-consuming. Unfortunately, despite being a common approach to mitigate labeling costs, existing weakly supervised learning methods struggle to adequately preserve supervised information for tail samples, resulting in a decline in accuracy for the tail classes. To alleviate this problem, we introduce a novel weakly supervised labeling setting called Reduced Label. The proposed labeling setting not only avoids the decline of supervised information for the tail samples, but also decreases the labeling costs associated with long-tailed data. Additionally, we propose an straightforward and highly efficient unbiased framework with strong theoretical guarantees to learn from these Reduced Labels. Extensive experiments conducted on benchmark datasets including ImageNet validate the effectiveness of our approach, surpassing the performance of state-of-the-art weakly supervised methods. Source code is available at \hrefhttps://github.com/WilsonMqz/LTRL https://github.com/WilsonMqz/LTRL

Annotating data for sensitive labels (e.g., disease, smoking) poses a potential threats to individual privacy in many real-world scenarios. To cope with this problem, we propose a novel setting to protect privacy of each instance, namely learning from concealed labels for multi-class classification. Concealed labels prevent sensitive labels from appearing in the label set during the label collection stage, which specifies none and some random sampled insensitive labels as concealed labels set to annotate sensitive data. In this paper, an unbiased estimator can be established from concealed data under mild assumptions, and the learned multi-class classifier can not only classify the instance from insensitive labels accurately but also recognize the instance from the sensitive labels. Moreover, we bound the estimation error and show that the multi-class classifier achieves the optimal parametric convergence rate. Experiments demonstrate the significance and effectiveness of the proposed method for concealed labels in synthetic and real-world datasets. Source code is available at https://github.com/WilsonMqz/CLF

Local feature extraction is one of the key characteristics of convolutional neural networks (CNNs). This study proposes an adaptive local feature enhancement (ALFE) model with a low‐frequency general appearance‐enhancement operator and a high‐frequency local detail enhancement operator to improve local features of CNNs. Through supervised training, the model could adaptively adjust enhancement parameters and achieve a global‐local enhancement of training images and CNNs. The performance of ALFE was first preliminarily evaluated with a self‐built CNN on CIFAR‐10 data set in different conditions of image augmentation and feature pooling. CNNs with ALFE could increase the top‐1 accuracy compared against CNNs in the same conditions, and nearly reach the same level of performance for different pooling approaches. With only two extra adjustable parameters, this model could effectively avoid overfitting, without affecting the convergence speed of CNN. In addition, the extra burden of network complexity could be neglected. Further, experiments of three existing popular CNNs (AlexNet, VGGNet and ResNet) with ALFE were carried out on dogs versus cats, Tiny ImageNet, and SVHN data sets, respectively. The results show that ALFE is feasible for the existing popular CNN models, improving their top‐1 accuracies without changing their convergence speeds.

A novel method based on rough sets (RS) and the affinity propagation (AP) clustering algorithm is developed to optimize a radial basis function neural network (RBFNN). First, attribute reduction (AR) based on RS theory, as a preprocessor of RBFNN, is presented to eliminate noise and redundant attributes of datasets while determining the number of neurons in the input layer of RBFNN. Second, an AP clustering algorithm is proposed to search for the centers and their widths without a priori knowledge about the number of clusters. These parameters are transferred to the RBF units of RBFNN as the centers and widths of the RBF function. Then the weights connecting the hidden layer and output layer are evaluated and adjusted using the least square method (LSM) according to the output of the RBF units and desired output. Experimental results show that the proposed method has a more powerful generalization capability than conventional methods for an RBFNN.