Feature dynamics have emerged as a critical topic about open-environment learning due to the instability of feature availability. While traditional feature evolution targets single-label tasks, multi-label learning is essential to accommodate the exploding annotation spaces. However, multi-label classification with incremental and decremental features is a crucial yet underexplored problem, which poses the challenge of preserving feature representations and label correlations from historical instances and simultaneously adapting to newly arriving streaming data. To address these issues, we propose a two-stage, one-pass learning approach termed MLID. It attempts to compress the informative content of vanished features into the domain of survived ones, facilitate the propagation of label dependencies via low-rank regularization of the classifier, and incorporate augmented features to construct an adaptive classification mechanism. Besides, we design optimization strategies for each stage and provide theoretical guarantees of convergence. Moreover, we establish the generalization error bound of MLID and demonstrate that the compactness of the trace norm and the reuse of models based on effective features can enhance the generalization performance. Finally, we extend it to multi-shot case and extensive experimental results validate the superiority of our MLID.

In multi-hop question answering (MHQA) tasks, Chain of Thought (CoT) improves the quality of generation by guiding large language models (LLMs) through multi-step reasoning, and Knowledge Graphs (KGs) reduce hallucinations via semantic matching. Outcome Reward Models (ORMs) provide feedback after generating the final answers but fail to evaluate the process for multi-step reasoning. Traditional Process Reward Models (PRMs) evaluate the reasoning process but require costly human annotations or rollout generation. While implicit PRM is trained only with outcome signals and derives step rewards through reward parameterization without explicit annotations, it is more suitable for multi-step reasoning in MHQA tasks. However, existing implicit PRM has only been explored for plain text scenarios. When adapting to MHQA tasks, it cannot handle the graph structure constraints in KGs and capture the potential inconsistency between CoT and KG paths. To address these limitations, we propose the DPRM (Dual Implicit Process Reward Model). It trains two implicit PRMs for CoT and KG reasoning in MHQA tasks. Both PRMs, namely KG-PRM and CoT-PRM, derive step-level rewards from outcome signals via reward parameterization without additional explicit annotations. Among them, KG-PRM uses preference pairs to learn structural constraints from KGs. DPRM further introduces a consistency constraint between CoT and KG reasoning steps, making the two PRMs mutually verify and collaboratively optimize the reasoning paths. We also provide a theoretical demonstration of the derivation of process rewards. Experimental results show that our method outperforms 13 baselines on multiple datasets with up to 16.6% improvement on Hit@1.

Partial label learning efficiently extracts accurate labels from weak supervision, where each instance has multiple candidate labels but only one is correct. Existing partial label learning methods typically employ a two-stage process and fall into a suboptimal solution in high-dimensional settings, lacking an effective feedback mechanism between feature selection and label disambiguation. Nevertheless, existing methods suffer from at least one of two issues, i.e., redundant features and the uncertainty of label disambiguation. To address these problems, we propose a unified and deep partial label feature selection method with CLuster-guided disAmbiguation and Structured Sparsity (CLASS), which simultaneously preserve discriminative features and promotes candidate label disambiguation by bidirectional optimization. Specifically, we integrate nonlinear deep networks to capture high-order semantic relations and feature interactions, while employing a linear sparse gating mechanism to preserve the label-specific, discriminative and interpretable features. Moreover, a cluster-guided disambiguation module is designed to enforce inter-class global separation and intra-class local cohesion via distance constraint and confidence penalty and dynamically align predicted labels with the semantics of class prototypes. Extensive experimental results validate the superiority and effectiveness of our proposed CLASS over state-of-the-art methods.

Accurate brain tumor segmentation plays an important role in clinical diagnosis, treatment planning, and therapeutic response monitoring. Multi-modal MRI provides complementary structural and functional information, but existing methods remain limited by their inadequate exploitation of cross-modal complementarity and their inability to effectively handle modality-specific disparities and redundant information. To address these challenges, this paper proposes a novel hierarchical teacher-student learning framework with adaptive cross-modal fusion. MRI modalities are grouped into teacher modalities (Flair and T1c) and student modalities (T2 and T1) based on their intrinsic tumor-related characteristics. Central to this framework is the Modality Guidance Module (MGM), which consists of two key components designed to achieve multi-modal feature distillation. Within MGM, the Modality Enhancement Module (MEM) extracts highly discriminative features from teacher modalities. While the Modality Fusion Module (MFM) leverages these features to guide and refine the learning of student modalities. To further capture inter-modal dependencies, a Cross-Modal Fusion Module (CMFM) is introduced to adaptively integrate complementary information across all modalities. Extensive experiments on the BraTS 2018, 2019 and 2020 datasets demonstrate that the proposed method achieves superior performance compared with state-of-the-art approaches. Beyond brain tumor segmentation, the hierarchical teacher-student paradigm and adaptive fusion strategy also hold potential for broader multi-modal image analysis tasks.

Accurate segmentation of brain tumors from multi-modal MRI is crucial for diagnosis and treatment planning. However, challenges such as severe class imbalance, modality-specific feature heterogeneity, and predictive uncertainty hinder reliable performance. In this work, we propose UTriGate-Net, a novel uncertainty-aware multi-modal brain tumor segmentation framework. First, we design a Triaxial Context Encoding (TCE) block that extracts anisotropic spatial features by applying directional convolutions along the axial, coronal, and sagittal planes, thereby enhancing 3D contextual representation. Second, we introduce a Gated Modality Fusion (GMF) module, which adaptively integrates complementary information across modalities through modality-specific gating weights that suppress redundancy while retaining salient features. Finally, to improve segmentation reliability, we develop an Uncertainty-Regularized Weighted Loss (URWL) that combines dynamic class-specific weighting to mitigate class imbalance with an entropy-based uncertainty penalty to encourage well-calibrated predictions. Experiments on the BraTS 2019 and 2020 datasets demonstrate that UTriGate-Net achieves superior segmentation accuracy and robustness, particularly in challenging subregions. Overall, the proposed framework offers a promising solution for reliable and precise brain tumor delineation in clinical practice.

Transfer subspace learning plays a critical role in unsupervised domain adaptation by establishing a shared embedding space where source domain data can be linearly reconstructed to match target domain distribution. While existing methods exploit the low-rank structures of reconstruction matrix, they frequently overlook the alignment of cross-domain joint probability distributions in the learned low-dimensional subspace. To address these challenges, we propose a novel non-convex transfer learning method named DATSL, which employs embedded distribution alignment. Our DATSL incorporates a non-convex regularizer to approximate low-rank constraints, capturing the complex characteristics of the rank function by minimizing top $k$ smallest singular values of reconstruction matrix. To align the joint distributions across domains, a category-aware joint distribution alignment mechanism extracts more discriminative representations and enhances subspace discriminability through label-informed covariance matching. Besides, DATSL is extended to a graph-based variant GDATSL, which incorporates manifold-preserving constraints via Laplacian regularization to maintain intrinsic data topology during knowledge transfer. Furthermore, we develop an efficient iterative optimization algorithm to solve our formulated nonconvex minimization problems with proved convergence. Extensive experimental results on several public datasets demonstrate the effectiveness of our proposed methods in comparison to other state-of-the-art approaches.

Imbalance in multi-domain multi-modal settings remains a significant challenge in real-world applications, such as fake news detection. Although existing methods have enhanced semantic representations and employed complex architectures to improve recognition performance, most of them focus on domain heterogeneity and modality sensitivity. However, these methods neglect the class imbalance, which severely impacts model robustness and generalization in open-world scenarios. Learning for such imbalanced multi-domain multi-modal data is crucial but rarely studied, particularly when the domain and modal of samples are imbalanced. To address these challenges, we propose a novel Imbalanced multi-domain multi-modal learning method with Expert Collaboration and Dynamic Fusion, named IECDF. Specifically, to alleviate inter-domain imbalance, we design an expert collaboration module with a domain-shared specific embedding structure and an improved gating strategy to enhance the discriminative power of the cross-domain features. Besides, a dynamic fusion mechanism based on Mamba-Former is designed to learn adaptive weights for each modality. Moreover, to tackle the intra-class imbalance problem, we adopt asymmetric re-weighted loss—DLINEX—to increase the contributions of minority class samples and learn the unbiased decision boundary. Extensive experimental results on various datasets validate the superiority of our proposed IECDF compared to state-of-the-art methods. Our code will be available at https://github.com/Yuchen-zh/IECDF .

Multi-view multi-label learning has become a research focus for describing objects with rich expressions and annotations. However, real-world data often contains numerous unlabeled instances, due to the high cost and technical limitations of manual labeling. This crucial problem involves three main challenges: i) How to extract advanced semantics from available views? ii) How to build a refined classification framework with limited labeled space? iii) How to provide more high-quality supervisory information? To address these problems, we propose a Semi-Supervised Multi-View Multi-Label Learning Method with View-Specific Transformer and Enhanced Pseudo-Label named SMVTEP. Specifically, Generative Adversarial Networks are employed to extract informative shared and specific representations and their consistency and distinctiveness are ensured through the adversarial mechanism and information theory based contrastive learning. Then we build specific classifiers for each extracted feature and apply instance-level manifold constraints to reduce bias across classifiers. Moreover, we design a transformer-style fusion approach that simultaneously captures the imbalance of expressive power among views, mapping effects on specific labels, and label dependencies by incorporating confidence scores and category semantics into the self-attention mechanism. Furthermore, after using Mixup for data augmentation, category-enhanced pseudo-labels are leveraged to improve the reliability of additional annotations by aligning the label distribution of unlabeled samples with the true distribution. Finally, extensive experimental results validate the effectiveness of SMVTEP against state-of-the-art methods.

Compositional visual question answering (Compositional VQA) needs to provide an answer to a compositional question, which requires the model to have advanced capabilities of multi-modal semantic understanding and logical reasoning. However, current VQA models mainly concentrate on enriching the visual representations of images and neglect the redundancy in the enriched information to bring some negative impacts. To enhance the value and availability of semantic features, we propose a novel core-to-global reasoning (CTGR) model for compositional VQA. The model first extracts both global features and core features from image and question through a feature embedding module. Then, to enhance the value of semantic features, we propose an information filtering module to align visual features and text features at the core semantic level and to filter out the redundancy carried by image and question features at the global semantic level, which can further strengthen cross-modal correlations. Besides, we design a novel core-to-global reasoning mechanism for multimodal fusion, which integrates content features from core learning and context features from global features for accurate answer predictions. Finally, extensive experimental results on GQA, GQA-sub, VQA2.0 and Visual7W demonstrate the effectiveness and superiority of CTGR.

Hallucinations in Large Vision-Language Models (LVLMs) significantly undermine their reliability, motivating researchers to explore the causes of hallucination. However, most studies primarily focus on the language aspect rather than the visual. In this paper, we address how LVLMs process visual information and whether this process causes hallucination. Firstly, we use the attention lens to identify the stages at which LVLMs handle visual data, discovering that the middle layers are crucial. Moreover, we find that these layers can be further divided into two stages: "visual information enrichment" and "semantic refinement" which respectively propagate visual data to object tokens and interpret it through text. By analyzing attention patterns during the visual information enrichment stage, we find that real tokens consistently receive higher attention weights than hallucinated ones, serving as a strong indicator of hallucination. Further examination of multi-head attention maps reveals that hallucination tokens often result from heads interacting with inconsistent objects. Based on these insights, we propose a simple inference-time method that adjusts visual attention by integrating information across various heads. Extensive experiments demonstrate that this approach effectively mitigates hallucinations in mainstream LVLMs without additional training costs.1

Incomplete features and label noise in multi-view multi-label data significantly undermine the reliability and performance, motivating researchers to explore the mechanism of representation and information recovery. However, learning for such dual deficiencies is crucial but rarely studied. In this paper, we propose a theory-inspired Deep Multi-View Multi-Label Learning method with Incomplete Views and Noisy Labels named DMMIvNL to address these problems. Specifically, to promote the synthesis of task-relevant shared information and preserve the distinctiveness of individual features from limited views, we have developed a feature extraction modular based on the information bottleneck theory, and formulated its theoretical upper bound into its objective. Meanwhile, we theoretically prove that minimizing the volume of the transition matrix ensures the statistical consistency with classifier training. Besides, a cycle-consistency estimation principle is proposed in the volume minimization network to improve the recognition stability of multi-label noise. Moreover, leveraging inherent real semantics information and label correlations are employed as model regularization to reduce the risk of excessive noise fitting. Finally, extensive experimental results validate the effectiveness and robustness of our DMMIvNL.

Multi-view classification (MVC) based on the Dempster-Shafer theory has gained significant recognition for its reliability in safety-critical applications. However, existing methods predominantly focus on providing confidence levels for decision outcomes without explaining the reasoning behind these decisions. Moreover, the reliance on first-order statistical magnitudes of belief masses often inadequately capture the intrinsic uncertainty within the evidence. To address these limitations, we propose a novel framework termed Trusted Multi-view Classification Constrained with Expert Knowledge (TMCEK). TMCEK integrates expert knowledge to enhance feature-level interpretability and introduces a distribution-aware subjective opinion mechanism to derive more reliable and realistic confidence estimates. The theoretical superiority of the proposed uncertainty measure over conventional approaches is rigorously established. Extensive experiments conducted on three multi-view datasets for sleep stage classification demonstrate that TMCEK achieves state-of-the-art performance while offering interpretability at both the feature and decision levels. These results position TMCEK as a robust and interpretable solution for MVC in safety-critical domains. The code is available at https://github.com/jie019/TMCEK_ICML2025.

Label shift is a prevalent phenomenon encountered in open environments, characterized by a notable discrepancy in the label distributions between the source (training) and target (test) domains, whereas the conditional distributions given the labels remain invariant. Existing label shift methods adopt a two-step strategy: initially computing the importance weight and subsequently utilizing it to calibrate the target outputs. However, this conventional strategy overlooks the intricate interplay between output adjustment and weight estimation. In this paper, we introduce a novel approach termed as One-step Label Shift Adaptation (OLSA). Our methodology jointly learns the predictive model and the corresponding weights through a bi-level optimization framework, with the objective of minimizing an upper bound on the target risk. To enhance the robustness of our proposed model, we incorporate a debiasing term into the upper-level classifier training and devise a regularization term for the lower-level weight estimation. Furthermore, we present theoretical analyses about the generalization bounds, offering guarantees for the model's performance. Extensive experimental results substantiate the efficacy of our proposal.

View missing remains a significant challenge in graph-based multi-view semi-supervised learning, hindering their real-world applications. To address this issue, traditional methods introduce a missing indicator matrix and focus on mining partial structure among existing samples in each view for label propagation (LP). However, we argue that these disregarded missing samples sometimes induce discontinuous local structures, i.e., sub-clusters, breaking the fundamental smoothness assumption in LP. Consequently, such a Sub-Cluster Problem (SCP) would distort graph fusion and degrade classification performance. To alleviate SCP, we propose a novel incomplete multi-view semi-supervised learning method, termed AGF-TI. Firstly, we design an adversarial graph fusion scheme to learn a robust consensus graph against the distorted local structure through a min-max framework. By stacking all similarity matrices into a tensor, we further recover the incomplete structure from the high-order consistency information based on the low-rank tensor learning. Additionally, the anchor-based strategy is incorporated to reduce the computational complexity. An efficient alternative optimization algorithm combining a reduced gradient descent method is developed to solve the formulated objective, with theoretical convergence. Extensive experimental results on various datasets validate the superiority of our proposed AGF-TI as compared to state-of-the-art methods. Code is available at https://github.com/ZhangqiJiang07/AGF_TI.

Multi-view multi-label learning typically suffers from dual data incompleteness due to limitations in feature storage and annotation costs. The interplay of hetero geneous features, numerous labels, and missing information significantly degrades model performance. To tackle the complex yet highly practical challenges, we propose a Theory-Driven Label-Specific Representation (TDLSR) framework. Through constructing the view-specific sample topology and prototype association graph, we develop the proximity-aware imputation mechanism, while deriving class representatives that capture the label correlation semantics. To obtain semantically distinct view representations, we introduce principles of information shift, inter action and orthogonality, which promotes the disentanglement of representation information, and mitigates message distortion and redundancy. Besides, label semantic-guided feature learning is employed to identify the discriminative shared and specific representations and refine the label preference across views. Moreover, we theoretically investigate the characteristics of representation learning and the generalization performance. Finally, extensive experiments on public datasets and real-world applications validate the effectiveness of TDLSR.

Evolutionary multi-view classification (EMVC) methods have gained wide recognition due to their adaptive mechanisms. Fitness evaluation (FE), which aims to calculate the classification performance of each individual in the population and provide reliable performance ranking for subsequent operations, is a core step in such methods. Its accuracy directly determines the correctness of the evolutionary direction. That is, when FE fails to correctly reflect the superiority-inferiority relationship among individuals, it will lead to confusion in individual performance ranking, which in turn misleads the evolutionary direction and results in trapping into local optima. This paper is the first to identify the aforementioned issue in the field of EMVC and call it as fitness evaluation bias (FEB). FEB may be caused by a variety of factors, and this paper approaches the issue from the perspective of view information content: existing methods generally adopt joint training strategies, which restrict the exploration of key information in views with low information content. This makes it difficult for multi-view model (MVM) to achieve optimal performance during convergence, which in turn leads to FE failing to accurately reflect individual performance rankings and ultimately triggering FEB. To address this issue, we propose an evolutionary multi-view classification via eliminating individual fitness bias (EFB-EMVC) method, which alleviates the FEB issue by introducing evolutionary navigators for each MVM, thereby providing more accurate individual ranking. Experimental results fully verify the effectiveness of the proposed method in alleviating the FEB problem, and the EMVC method equipped with this strategy exhibits more superior performance compared with the original EMVC method. (The code is available at https://github.com/LiShuailzn/Neurips-2025-EFB-EMVC)

Scene graph generation (SGG) establishes a structured representation between multiple objects by exploring their relationship for visual perception and reasoning tasks. Existing SGG methods often fit the relationships' distribution by introducing language prior or statistical knowledge. However, the relationships should be the semantic reflection of the interaction between objects, rather than the statistical dependency between their categories. To solve this problem, we propose a novel Causal Features Enhancement Network (CFEN) to mine the essential semantic features between objects and relationships. Specifically, by decomposing the object features into class-generic and object-specific components, the causal graph framework is designed to analyze these existing SGG methods. To measure the influence of object-specific features for relationship recognition, we construct the counterfactual training framework for computing the difference between fact and counterfactual logits. Besides, to strengthen the role of object-specific features and learn the interaction between objects, a distribution matching loss is proposed to compute the KL divergence between counterfactual outputs and standard difference distributions and modulate the relations predictions. Finally, compared with the current state-of-the-art methods, the extensive experimental results on VG150 and VrR-VG datasets demonstrate the effectiveness and superiority of our proposed CFEN.

Due to the efficiency of integrating semantic consensus and complementary information across different views, multi-view classification methods have attracted much attention in recent years. However, multi-view data often suffers from both the miss of view features and insufficient label information, which significantly decrease the performance of traditional multi-view classification methods in practice. Learning for such simultaneous lack of feature and label is crucial but rarely studied. To tackle these problems, we propose a novel Deep Incomplete Multi-view Learning Network (DIMvLN) by incorporating graph networks and semi-supervised learning in this paper. Specifically, DIMvLN firstly designs the deep graph networks to effectively recover missing data with assigning pseudo-labels of large amounts of unlabeled instances and refine the incomplete feature information. Meanwhile, to enhance the label information, a novel pseudo-label generation strategy with the similarity constraints of unlabeled instances is proposed to exploit additional supervisory information and guide the completion module to preserve more semantic information of absent multi-view data. Besides, we design view-specific representation extractors with the autoencoder structure and contrastive loss to learn high-level semantic representations for each view, promote cross-view consistencies and augment the separability between different categories. Finally, extensive experimental results demonstrate the effectiveness of our DIMvLN, attaining noteworthy performance improvements compared to state-of-the-art competitors on several public benchmark datasets. Code will be available at GitHub.

Due to the explosive growth in data sources and label categories, multi-view multi-label learning has garnered widespread attention. However, multi-view multi-label data often exhibits incomplete features and a huge number of unlabeled instances, due to the technical limitations and high cost of manual labeling in practice. Learning for such simultaneous missing of view features and labels is crucial but rarely studied, particularly when the labeled samples are limited. In this paper, we tackle this problem by proposing a novel Deep Incomplete Multi-View Semi-Supervised Multi-Label Learning method (DIMvSML). Specifically, to improve high-level representations of missing features, deep graph network is firstly employed to recover the feature information with structural similarity relations. Meanwhile, we design the structure-specific deep feature extractors to obtain discriminative information and preserve the cross-view consistency for the recovered data with instance-level contrastive loss. Furthermore, to eliminate the bias of the estimate of the risk that the semi-supervised multi-label methods minimise, we design a safe estimate framework with an unbiased loss and improve its empirical performance by using pseudo-labels of unlabeled data. Besides, we provide both the theoretical proof of better estimate variance and the intuitive explanation of our debiased framework. Finally, extensive experimental results on public datasets validate the superiority of DIMvSML compared with state-of-the-art methods.

Multi-instance multi-label learning (MIML), which deals with objects with complex structures and multiple semantics, plays a crucial role in various fields. In practice, the naturally skewed label distribution and label dependence contribute to the issue of label imbalance in MIML, which is crucial but rarely studied. Most existing MIML methods often produce biased models due to the ignorance of inter-class variations in imbalanced data. To address this issue, we propose a novel imbalanced multi-instance multi-label learning method named IMIMLC, based on the error-correcting coding ensemble and an adaptive threshold strategy. Specifically, we design a feature embedding method to extract the structural information of each object via Fisher vectors and eliminate inexact supervision. Subsequently, to alleviate the disturbance caused by the imbalanced distribution, a novel ensemble model is constructed by concatenating the error-correcting codes of randomly selected subtasks. Meanwhile, IMIMLC trains binary base classifiers on small-scale data blocks partitioned by our codes to enhance their diversity and then learns more reliable results to improve model robustness for the imbalance issue. Furthermore, IMIMLC adaptively learns thresholds for each individual label by margin maximization, preventing inaccurate predictions caused by the semantic discrepancy across many labels and their unbalanced ratios. Finally, extensive experimental results on various datasets validate the effectiveness of IMIMLC against state-of-the-art approaches.

With the advent of vast data collection ways, data are often with multiple modalities or coming from multiple sources. Traditional multiview learning often assumes that each example of data appears in all views. However, this assumption is too strict in some real applications such as multisensor surveillance system, where every view suffers from some data absent. In this article, we focus on how to classify such incomplete multiview data in semisupervised scenario and a method called absent multiview semisupervised classification (AMSC) has been proposed. Specifically, partial graph matrices are constructed independently by anchor strategy to measure the relationships among between each pair of present samples on each view. And to obtain unambiguous classification results for all unlabeled data points, AMSC learns view-specific label matrices and a common label matrix simultaneously. AMSC measures the similarity between pair of view-specific label vectors on each view by partial graph matrices, and consider the similarity between view-specific label vectors and class indicator vectors based on the common label matrix. To characterize the contributions of different views, the p th root integration strategy is adopted to incorporate the losses of different views. By further analyzing the relation between the p th root integration strategy and exponential decay integration strategy, we develop an efficient algorithm with proved convergence to solve the proposed nonconvex problem. To validate the effectiveness of AMSC, comparisons are made with some benchmark methods on real-world datasets and in the document classification scenario as well. The experimental results demonstrate the advantages of our proposed approach.

Clustering is a fundamental and important step in many image processing tasks, such as face recognition and image segmentation. The performance of clustering can be largely enhanced if relevant weak supervision information is appropriately exploited. To achieve this goal, in this paper, we propose the Compound Weakly Supervised Clustering (CSWC) method. Concretely, CSWC incorporates two types of widely available and easily accessed weak supervision information from the label and feature aspects, respectively. To be specific, at the label level, the pairwise constraints are utilized as a kind of typical weak label supervision information. At the feature level, the partial instances collected from multiple perspectives have internal consistency and they are regarded as weak structure supervision information. To achieve a more confident clustering partition, we learn a unified graph with its similarity matrix to incorporate the above two types of weak supervision. On one hand, this similarity matrix is constructed by self-expression across the partial instances collected from multiple perspectives. On the other hand, the pairwise constraints, i.e., must-links and cannot-links, are considered by formulating a regularizer on the similarity matrix. Finally, the clustering results can be directly obtained according to the learned graph, without performing additional clustering techniques. Besides evaluating CSWC on 7 benchmark datasets, we also apply it to the application of face clustering in video data since it has vast application potentiality. Experimental results demonstrate the effectiveness of our algorithm in both incorporating compound weak supervision and identifying faces in real applications.

Scene graph generation (SGG) is one of the hottest topics in computer vision and has attracted many interests since it provides rich semantic information between objects. In practice, the SGG datasets are often dual imbalanced, presented as a large number of backgrounds and rarely few foregrounds, and highly skewed foreground relationships categories (i.e., the long-tailed distribution). How to tackle this dual imbalanced problem is crucial but rarely studied in literature. Existing methods only consider the long-tailed distribution of foregrounds classes and ignore the background-foreground imbalance in SGG, which results in a biased model and prevents it from being applied in the downstream tasks widely. To reduce its side effect and make the contributions of different categories equally, we propose a novel debiased SGG method (named DSDI) by incorporating biased resistance loss and causal intervention tree. We first deeply analyze the potential causes of dual imbalanced problem in SGG. Then, to learn more discriminate representation of the foreground by expanding the foreground features space, the biased resistance loss decouples the background classification from foreground relationship recognition. Meanwhile, a causal graph of content and context is designed to remove the context bias and learn unbiased relationship features via casual intervention tree. Extensive experimental results on two extremely imbalanced datasets: VG150 and VrR-VG, demonstrate our DSDI outperforms other state-of-the-art methods. All our models will be available in https://github.com/zhouhao0515/unbiasedSGG-DSDI.

In image processing, images are usually composed of partial views due to the uncertainty of collection and how to efficiently process these images, which is called incomplete multi-view learning, has attracted widespread attention. The incompleteness and diversity of multi-view data enlarges the difficulty of annotation, resulting in the divergence of label distribution between the training and testing data, named as label shift. However, existing incomplete multi-view methods generally assume that the label distribution is consistent and rarely consider the label shift scenario. To address this new but important challenge, we propose a novel framework termed as Incomplete Multi-view Learning under Label Shift (IMLLS). In this framework, we first give the formal definitions of IMLLS and the bidirectional complete representation which describes the intrinsic and common structure. Then, a multilayer perceptron which combines the reconstruction and classification loss is employed to learn the latent representation, whose existence, consistency and universality are proved with the theoretical satisfaction of label shift assumption. After that, to align the label distribution, the learned representation and trained source classifier are used to estimate the importance weight by designing a new estimation scheme which balances the error generated by finite samples in theory. Finally, the trained classifier reweighted by the estimated weight is fine-tuned to reduce the gap between the source and target representations. Extensive experimental results validate the effectiveness of our algorithm over existing state-of-the-arts methods in various aspects, together with its effectiveness in discriminating schizophrenic patients from healthy controls.

Online learning is a fundamental paradigm for learning from continuous data stream. Tradition online learning approaches usually assume that the feature space of data stream is fixed and the incoming instance can always get the true label after making its prediction. However, in many real-world applications, such as the personalized recommender systems, the feature space may keep expanding due to the accumulation of user behaviors. Besides, we may only get bandit feedback, i.e., we only know whether the prediction is correct or not. To solve this important but rarely studied problem, we propose a novel algorithm LIFBF, together with its two variants LIFBF-I and LIFBF-II, to learn from data stream with incremental feature space and bandit feedback. Specifically, when an instance arrives with augmented features, we first utilize the exploration-exploitation strategy to guess its best label, then, a new loss function considering both bandit feedback and guessed label is proposed. Finally, we design a highly dynamic multi-class classifier, which updates the shared and augmented features by adopting the passive-aggressive rule and structural risk minimization principle, respectively. We theoretically analyze the cumulative loss bound of LIFBF. Besides, empirical studies on various datasets further validate the effectiveness of our proposed algorithms.

The scarcity of labels is common and great challenge in traditional supervised learning. Semi-supervised learning (SSL) leverages unlabeled samples to alleviate the absence of label information. Similar with annotation, label proportion is another type of prior information and plays a significant role in classification tasks. Compared with the acquisition of labels, label proportion can be obtained more easily. For example, only a small number of patients have been diagnosed with or not with cancers in hospital database, while the proportion with cancer can be generally estimated by historical records. How to incorporate such prior information of label proportion is crucial but rarely studied in literature. Traditional SSL methods often ignore this prior information and will lead to performance degradation inevitably. To solve this problem, we propose a novel SSL with Label Proportion (SSLLP). Our approach encourages to preserve label consistency and label proportion by imposing the cardinality bound constraints. Our formulated problem equals to a mixed-integer constrained submodular minimization and it is difficult to be solved directly. Therefore, we transformed the original problem into a convex one by Lov$\acute{\text{a}}$sz extension and designed an efficient solving algorithm. Extensive experimental results present the improved performance of our method over several state-of-the-art methods.

Multiple Instance Learning (MIL) is a widely studied learning paradigm which arises from real applications. Existing MIL methods have achieved prominent performances under the premise of plenty annotation data. Nevertheless, sufficient labeled data is often unattainable due to the high labeling cost. For example, the task in web image identification is to find similar samples among a large size of unlabeled dataset through a small number of provided target pictures. This leads to a particular scenario of Multiple Instance Learning with insufficient Positive and superabundant Unlabeled data (PU-MIL), which is a hot research topic in MIL recently. In this paper, we propose a novel method called Multiple Instance Learning with Bi-level Embedding (MILBLE) to tackle PU-MIL problem. Unlike other PU-MIL method using only simple single-level mapping, the bi-level embedding strategy are designed to customize specific mapping for positive and unlabeled data. It ensures the characteristics of key instance are not erased. Moreover, the weighting measure adopted in positive data can extracts the uncontaminated information of true positive instances without interference from negative ones. Finally, we minimize the classification error loss of mapped examples based on class-prior probability to train the optimal classifier. Experimental results show that our method has better performance than other state-of-the-art methods.

Partial multi-view clustering has attracted various attentions from diverse fields. Most existing methods adopt separate steps to obtain unified representations and extract clustering indicators. This separate manner prevents two learning processes to negotiate to achieve optimal performance. In this paper, we propose the Joint Representation Learning and Clustering (JRLC) framework to address this issue. The JRLC framework employs representation matrices to extract view-specific clustering information directly from the presence of partial similarity matrices, and rotates them to learn a common probability label matrix simultaneously, which connects representation learning and clustering seamlessly to achieve better clustering performance. Under the guidance of JRLC framework, several new incomplete multi-view clustering methods can be developed by extending existing single-view graph-based representation learning methods. For illustration, within the framework, we propose two specific methods, JRLC with spectral embedding (JRLC-SE) and JRLC via integrating nonnegative embedding and spectral embedding (JRLC-NS). Two iterative algorithms with guaranteed convergence are designed to solve the resultant optimization problems of JRLC-SE and JRLC-NS. Experimental results on various datasets and news topic clustering application demonstrate the effectiveness of the proposed algorithms.

The relationships and interactions between objects contain rich semantic information, which plays a crucial role in scene understanding. Existing methods do not attach great importance to the expression of relational features. To tackle this problem, we propose a novel Relationship-aware Primal-Dual Graph Attention Network (RPDGAT) to extract the comprehensive semantic features of objects and explore the sparse graph inference for scene graph generation. RPDGAT mines the inherent attributes and the relationships between objects by fusing multiple features, e.g. appearance, spatial, and category features. After feature extraction, we design a trainable relationship distance measure network to construct the robust and sparse graph structure for efficient graphical message passing. Moreover, it can preserve the contextual cues and neighboring dependency for objects and relationships from the interaction between primal and dual graphs. Extensive experimental results present the improved performance of our method over several state-of-the-art methods on the visual genome datasets.

Transfer subspace learning aims to learn robust subspace for the target domain by leveraging knowledge from the source domain. The traditional methods often adopt the convex norm to approximate the original sparse and low-rank constraints, which make the optimization problem be easily solved. However, such relax approximation leads to the performance deviation of the original non-convex model. In this paper, we propose a novel Non-convex Transfer Subspace Learning~(NTSL) method to provide a tighter approximation to the original sparse and low-rank constraints. Specifically, we design an objective function that leverages the Schatten p-norm and ℓ_2, p-norm to preserve the structure between the source and target domains. With Schatten p-norm, the objective function better approximates the rank minimization problem than the nuclear norm and preserves the structure of domains. Besides, the ℓ_2, p-norm can reduce the effect of noise and improve the robustness to outliers. Meanwhile, we develop an efficient algorithm to solve the non-convex minimization problem. Extensive experimental results on cross-domain tasks show the effectiveness of our proposed method.

High-dimensional non-Gaussian data are ubiquitous in many real applications. Face recognition is a typical example of such scenarios. The sampled face images of each person in the original data space are more closely located to each other than to those of the same individuals due to the changes of various conditions like illumination, pose variation, and facial expression. They are often non-Gaussian and differentiating the importance of each data point has been recognized as an effective approach to process the high-dimensional non-Gaussian data. In this paper, to embed non-Gaussian data well, we propose a novel unified framework named adaptive discriminative analysis (ADA), which combines the sample's importance measurement and subspace learning in a unified framework. Therefore, our ADA can preserve the within-class local structure and learn the discriminative transformation functions simultaneously by minimizing the distances of the projected samples within the same classes while maximizing the between-class separability. Meanwhile, an efficient method is developed to solve our formulated problem. Comprehensive analyses, including convergence behavior and parameter determination, together with the relationship to other related approaches, are as well presented. Systematical experiments are conducted to understand the work of our proposed ADA. Promising experimental results on various types of real-world benchmark data sets are provided to examine the effectiveness of our algorithm. Furthermore, we have also evaluated our method in face recognition. They all validate the effectiveness of our method on processing the high-dimensional non-Gaussian data.

Feature selection plays a significant role in dealing with high-dimensional data to avoid the curse of dimensionality. In many real applications, like video semantic recognition, handling few labeled and large unlabeled data samples from the same population is a recently addressed challenge in feature selection. To solve this problem, we propose a novel semi-supervised feature selection method via insensitive sparse regression (ISR). Specifically, we compute the soft label matrix by the special label propagation, which can predict the labels of the unlabeled data. To guarantee the robustness of ISR to the false labeled instances or outliers, we propose Insensitive Regression Model (IRM) by capped$l_2$-$l_p$-norm loss. The soft label is imposed as the weights of IRM to fully utilize the label information. Meanwhile, to perform feature selection, we incorporate$l_{2,q}$-norm regularizer with IRM as the structural sparsity constraint when$0 < q\leq 1$. Moreover, we put forward an effective approach for solving the formulated non-convex optimization problem. We analyze the performance of convergence rigorously and discuss the parameter determination problem. Extensive experimental results on several public data sets verify the effectiveness of our proposed algorithm in comparison with the state-of-art feature selection methods. Finally, we apply our method to video semantic recognition successfully.

The genome-wide association study (GWAS) is a popular approach to identify disease-associated genetic factors for Alzhemer's Disease (AD). However, it remains challenging because of the small number of samples, very high feature dimensionality and complex structures. To accurately identify genetic risk factors for AD, we propose a novel method based on an in-depth exploration of the hierarchical structure among the features and the commonality across related tasks. Specifically, we first extract and encode the tree hierarchy among features; then, we integrate the tree structures with multi-task feature learning (MTFL) to learn the shared features-that are predictive of AD-among related tasks simultaneously. Thus, we can unify the strength of both the prior structure information and MTFL to boost the prediction performance. However, due to the highly complex regularizer that encodes the tree structure and the extremely high feature dimensionality, the learning process can be computationally prohibitive. To address this, we further develop a novel safe screening rule to quickly identify and remove the irrelevant features before training. Experiment results demonstrate that the proposed approach significantly outperforms the state-of-the-art in detecting genetic risk factors of AD and the speedup gained by the proposed screening can be several orders of magnitude.

Functional annotation of human genes is fundamentally important for understanding the molecular basis of various genetic diseases. A major challenge in determining the functions of human genes lies in the functional diversity of proteins, that is, a gene can perform different functions as it may consist of multiple protein coding isoforms (PCIs). Therefore, differentiating functions of PCIs can significantly deepen our understanding of the functions of genes. However, due to the lack of isoform-level gold-standards (ground-truth annotation), many existing functional annotation approaches are developed at gene-level. In this paper, we propose a novel approach to differentiate the functions of PCIs by integrating sparse simplex projection---that is, a nonconvex sparsity-inducing regularizer---with the framework of multi-instance learning (MIL). Specifically, we label the genes that are annotated to the function under consideration as positive bags and the genes without the function as negative bags. Then, by sparse projections onto simplex, we learn a mapping that embeds the original bag space to a discriminative feature space. Our framework is flexible to incorporate various smooth and non-smooth loss functions such as logistic loss and hinge loss. To solve the resulting highly nontrivial non-convex and non-smooth optimization problem, we further develop an efficient block coordinate descent algorithm. Extensive experiments on human genome data demonstrate that the proposed approaches significantly outperform the state-of-the-art methods in terms of functional annotation accuracy of human PCIs and efficiency.

For many image processing and computer vision problems, data points are in matrix form. Traditional methods often convert a matrix into a vector and then use vector-based approaches. They will ignore the location of matrix elements and the converted vector often has high dimensionality. How to select features for 2D matrix data directly is still an uninvestigated important issue. In this paper, we propose an algorithm named sparse matrix regression (SMR) for direct feature selection on matrix data. It employs the matrix regression model to accept matrix as input and bridges each matrix to its label. Based on the intrinsic property of regression coefficients, we design some sparse constraints on the coefficients to perform feature selection. An effective optimization method with provable convergence behavior is also proposed. We reveal that the number of regression vectors can be regarded as a tradeoff parameter to balance the capacity of learning and generalization in essence. To examine the effectiveness of SMR, we have compared it with several vector-based approaches on some benchmark data sets. Furthermore, we have also evaluated SMR in the application of scene classification. They all validate the effectiveness of our method.