Benefiting from the effective exploitation of the high-order correlations across multiple views, tensor-based multi-view clustering (TMVC) has garnered considerable attention in recent years. Nevertheless, prior TMVC techniques commonly involve assembling multiple view-specific spatial similarity graphs into a three-dimensional tensor, overlooking the intrinsic topological structure essential for precise clustering of data within a manifold. Additionally, mainstream techniques are constrained by equally shrinking all singular values to recover a low-rank tensor, limiting their capacity to distinguish significant variations among different singular values. In this investigation, we present an innovative TMVC framework termed toPology-driven multi-view clustering viA refined teNsorial sigmoiD rAnk minimization (PANDA ). Specifically, PANDA extracts view-specific topological structures from Euclidean graphs and intricately integrates them into a low-rank three-dimensional tensor, facilitating the concurrent utilization of intra-view topological connectivity and inter-view high-order correlations. Moreover, we develop a refined sigmoid function as the tighter surrogate to tensor rank, enabling the exploration of significant information of heterogeneous singular values. Meanwhile, the topological structures are merged into a unified structure with varying weights, associated with a connectivity constraint, empowering the significant divergence among views and the explicit cluster structure of the target graph are simultaneously leveraged. Extensive experiments demonstrate the superiority of PANDA, outperforming SOTA methods.

The Unaligned Multi-view Clustering (UMC) problem is currently receiving widespread attention, focusing on clustering unaligned multi-view data generated in real-world applications. Although some algorithms have emerged to address this issue, there still exist the following drawbacks: 1) The fully unknown correspondence of samples across views can significantly limit the exploration of consistent clustering structure. 2) The fixed representation space makes it difficult to mine the comprehensive information in the original data. 3) Unbiased tensor rank approximation is desired to capture the high-order correlation among different views. To address these issues, we proposed a novel UMC framework termed Tensorized Unaligned Multi-view Clustering with Multi-scale Representation Learning (TUMCR). Specifically, TUMCR designs a multi-scale representation learning and alignment framework, which constructs multi-scale representation spaces to comprehensively explore the unknown correspondence across views. Then, a tensorial multi-scale fusion module is proposed to fuse multi-scale representations and explore the high-order correlation hidden in different views, which utilizes the Enhanced Tensor Rank (ETR) to learn the low-rank structure. Furthermore, TUMCR is solved by an efficient algorithm with good convergence. Extensive experiments on different types of datasets demonstrate the effectiveness and superiority of our TUMCR compared with state-of-the-art methods. Our code is publicly available at: https://github.com/jijintian/TUMCR.

Benefiting from the effective exploration of the valuable topological pair-wise relationship of data points across multiple views, multi-view subspace clustering (MVSC) has received increasing attention in recent years. However, we observe that existing MVSC approaches still suffer from two limitations that need to be further improved to enhance the clustering effectiveness. Firstly, previous MVSC approaches mainly prioritize extracting multi-view consistency, often neglecting the cross-view discrepancy that may arise from noise, outliers, and view-inherent properties. Secondly, existing techniques are constrained by their reliance on pair-wise sample correlation and pair-wise view correlation, failing to capture the high-order correlations that are enclosed within multiple views. To address these issues, we propose a novel MVSC framework via joiNt crOss-view discrepancy discOvery anDhigh-order correLation dEtection (NOODLE), seeking an informative target subspace representation compatible across multiple features to facilitate the downstream clustering task. Specifically, we first exploit the self-representation mechanism to learn multiple view-specific affinity matrices, which are further decomposed into cohesive factors and incongruous factors to fit the multi-view consistency and discrepancy, respectively. Additionally, an explicit cross-view sparse regularization is applied to incoherent parts, ensuring the consistency and discrepancy to be precisely separated from the initial subspace representations. Meanwhile, the multiple cohesive parts are stacked into a three-dimensional tensor associated with a tensor-Singular Value Decomposition (t-SVD) based weighted tensor nuclear norm constraint, enabling effective detection of the high-order correlations implicit in multi-view data. Our proposed method outperforms state-of-the-art methods for multi-view clustering on six benchmark datasets, demonstrating its effectiveness.

Redundant Label Learning (RLL) aims at inducing a robust model from training data, where each example is associated with a set of candidate labels, among which some of them are incorrect. Most existing approaches deal with such problem by disambiguating the candidate labels first and then inducing the predictive model from the disambiguated data. However, these approaches only focus on disambiguation for each instance’ candidate label set, while the global label context tends to be ignored. Meanwhile, these approaches usually induce the objective model by directly utilizing the original feature information, which may lead to the model overfitting due to high-dimensional redundant features. To tackle the above issues, we propose a novel feature S ubspac E R epresentation and label G lobal Disambiguat IO n ( SERGIO ) approach, which improves the generalization ability of the learning system from the perspective of both feature space and label space. Specifically, we project the original high-dimensional feature space into a low-dimensional subspace, where the projection matrix is regularized with an orthogonality constraint to make the subspace more compact. Meanwhile, we introduce a label confidence matrix and constrain it with ℓ 1 -norm and trace-norm regularization simultaneously, which are utilized to explore global label correlations and further well in accordance with the nature of single-label classification and multi-label classification problem, respectively. Extensive experiments on both single-label and multi-label RLL datasets demonstrate that our proposed method achieves competitive performance against state-of-the-art approaches.

Partial label learning (PLL) aims to learn a robust multi-class classifier from the ambiguous data, where each instance is given with several candidate labels, among which only one label is real. Most existing methods usually cope with such problem by utilizing a feature similarity graph to conduct label disambiguation. However, these methods construct the feature graph by only employing original features, while the influences of latent outliers and the contributions of label space are regrettably ignored. To tackle these issues, in this article, we propose aPrior KnOwledge ConsTrainedAdaptiveGraph FramEwork (POTAGE) for partial label learning, which utilizes an adaptive graph fused with label information to accurately describe the instance relationship and guide the desired model training. Compared with the feature-induced fixed graph, the adaptive graph is deemed to be more robust and accurate to reveal the intrinsic manifold structure within the data, and the embedding label information is expected to effectively alleviate the label ambiguities and enlarge the gap of label confidences between two instances from different classes. Extensive experiments demonstrate that POTAGE achieves state-of-the-art performance.

Multi-view clustering, which aims at boosting the clustering performance by leveraging the individual information and the common information of multi-view data, has gained extensive consideration in recent years. However, most existing multi-view clustering algorithms either focus on extracting the multi-view individuality or emphasize on exploring the multi-view commonality, neither of which can fully utilize the comprehensive information from multiple views. To this end, we propose a novel algorithm named V iew-specific and C onsensus G raph A lignment (VCGA) for multi-view clustering, which simultaneously formulates the multi-view individuality and the multi-view commonality into a unified framework to effectively partition data points. To be specific, the VCGA model constructs the view-specific graphs and the shared graph from original multi-view data and hidden latent representation, respectively. Furthermore, the view-specific graphs of different views and the consensus graph are aligned into an informative target graph, which is employed as a crucial input to the standard spectral clustering method for clustering. Extensive experimental results on six benchmark datasets demonstrate the superiority of our method against other state-of-the-art clustering algorithms.

Graph-based Multi-View Clustering (GMVC) has received extensive attention due to its ability to capture the neighborhood relationship among data points from diverse views. However, most existing approaches construct similarity graphs from the original multi-view data, the accuracy of which heavily and implicitly relies on the quality of the original multiple features. Moreover, previous methods either focus on mining the multi-view commonality or emphasize on exploring the multi-view individuality, making the rich information contained in multiple features cannot be effectively exploited. In this work, we design a novel GMVC framework via c O mmo N ality and I ndividuality disc O vering in late N t subspace ( ONION ), seeking for a robust and discriminative subspace representation compatible across multiple features for GMVC. To be specific, our method simultaneously formulates the unsupervised sparse feature selection and the robust subspace extraction, as well as the target graph learning in a unified optimization model, which can help the learning of the discriminative subspace representation and the target graph in a mutual reinforcement manner. Meanwhile, we manipulate the target graph by an explicit structural penalty, rendering the connected components in the graph directly reveal clusters. Experimental results on seven benchmark datasets demonstrate the effectiveness of our proposed method.

The anchor graph structure has been widely used to speed up large-scale multi-view clustering and exhibited promising performance. How to effectively integrate the anchor graphs on multiple views to achieve enhanced clustering performance still remains a challenging task. Existing fusing strategies ignore the structure diversity among anchor graphs and restrict the anchor generation to be same on different views, which degenerates the representation ability of corresponding fused consensus graph. To overcome these drawbacks, we propose a novel structural fusion framework to integrate the multi-view anchor graphs for clustering. Different from traditional integration strategies, we merge the anchors and edges of all the view-specific anchor graphs into a single graph for the structural optimal graph learning. Benefiting from the structural fusion strategy, the anchor generation of each view is not forced to be same, which greatly improves the representation capability of the target structural optimal graph, since the anchors of each view capture the diverse structure of different views. By leveraging the potential structural consistency among each anchor graph, a connectivity constraint is imposed on the target graph to indicate clusters directly without any post-processing such as k -means in classical spectral clustering. Substantial experiments on real-world datasets are conducted to verify the superiority of the proposed method, as compared with the state-of-the-arts over the clustering performance and time expenditure.

Graph-based multi-view learning has attracted much attention due to the efficacy of fusing the information from different views. However, most of them exhibit high computational complexity. We propose an anchor-based bipartite graph embedding approach to accelerate the learning process. Specifically, different from existing anchor-based methods where anchors are obtained from key samples by clustering or weighted averaging strategies, in this article, the anchors are learned in a principled fashion which aims at constructing a distance-preserving embedding for each view from samples to their representations, whose elements are the weights of the edges linking corresponding samples and anchors. In addition, the consistency among different views can be explored by imposing a low-rank constraint on the concatenated embedding representations. We further design a concise yet effective feature collinearity guided feature selection scheme to learn tight multi-label classifiers. The objective function is optimized in an alternating optimization fashion. Both theoretical analysis and experimental results on different multi-label image datasets verify the effectiveness and efficiency of the proposed method.

Recently, concerning the challenge of collecting large-scale explicitly annotated videos, weakly supervised video object segmentation (WSVOS) using video tags has attracted much attention. Existing WSVOS approaches follow a general pipeline including two phases, i.e., a pseudo masks generation phase and a refinement phase. To explore the intrinsic property and correlation buried in the video frames, most of them focus on the later phase by introducing optical flow as temporal information to provide more supervision. However, these optical flow-based studies are greatly affected by illumination and distortion and lack consideration of the discriminative capacity of multi-level deep features. In this article, with the goal of capturing more effective temporal information and investigating a temporal information fusion strategy accordingly, we propose a unified WSVOS model by adopting a two-branch architecture with a multi-level cross-branch fusion strategy, named as dual-attention cross-branch fusion network (DACF-Net). Concretely, the two branches of DACF-Net, i.e., a temporal prediction subnetwork (TPN) and a spatial segmentation subnetwork (SSN), are used for extracting temporal information and generating predicted segmentation masks, respectively. To perform the cross-branch fusion between TPN and SSN, we propose a dual-attention fusion module that can be plugged into the SSN flexibly. We also pose a cross-frame coherence loss (CFCL) to achieve smooth segmentation results by exploiting the coherence of masks produced by TPN and SSN. Extensive experiments demonstrate the effectiveness of proposed approach compared with the state-of-the-arts on two challenging datasets, i.e., Davis-2016 and YouTube-Objects.

Semantic image synthesis is a new rising and challenging vision problem accompanied by the recent promising advances in generative adversarial networks. The existing semantic image synthesis methods only consider the global information provided by the semantic segmentation mask, such as class label, global layout, and location, so the generative models cannot capture the rich local fine-grained information of the images (e.g., object structure, contour, and texture). To address this issue, we adopt a multi-scale feature fusion algorithm to refine the generated images by learning the fine-grained information of the local objects. We propose OA-GAN, a novel object-attention generative adversarial network that allows attention-driven, multi-fusion refinement for fine-grained semantic image synthesis. Specifically, the proposed model first generates multi-scale global image features and local object features, respectively, then the local object features are fused into the global image features to improve the correlation between the local and the global. In the process of feature fusion, the global image features and the local object features are fused through the channel-spatial-wise fusion block to learn ‘what’ and ‘where’ to attend in the channel and spatial axes, respectively. The fused features are used to construct correlation filters to obtain feature response maps to determine the locations, contours, and textures of the objects. Extensive quantitative and qualitative experiments on COCO-Stuff, ADE20K and Cityscapes datasets demonstrate that our OA-GAN significantly outperforms the state-of-the-art methods.

Partial Label Learning (PLL) aims to learn from the data where each training example is associated with a set of candidate labels, among which only one is correct. The key to deal with such problem is to disambiguate the candidate label sets and obtain the correct assignments between instances and their candidate labels. In this paper, we interpret such assignments as instance-to-label matchings, and reformulate the task of PLL as a matching selection problem. To model such problem, we propose a novel Graph Matching based Partial Label Learning (GM-PLL) framework, where Graph Matching (GM) scheme is incorporated owing to its excellent capability of exploiting the instance and label relationship. Meanwhile, since conventional one-to-one GM algorithm does not satisfy the constraint of PLL problem that multiple instances may correspond to the same label, we extend a traditional one-to-one probabilistic matching algorithm to the many-to-one constraint, and make the proposed framework accommodate to the PLL problem. Moreover, we also propose a relaxed matching prediction model, which can improve the prediction accuracy via GM strategy. Extensive experiments on both artificial and real-world data sets demonstrate that the proposed method can achieve superior or comparable performance against the state-of-the-art methods.

Partial Multi-Label learning (PML) learns from the ambiguous data where each instance is associated with a candidate label set, where only a part is correct. The key to solve such problem is to disambiguate the candidate label sets and identify the correct assignments between instances and their ground-truth labels. In this paper, we interpret such assignments as instance-to-label matchings, and formulate the task of PML as a matching selection problem. To model such problem, we propose a novel grapH mAtching based partial muLti-label lEarning (HALE) framework, where Graph Matching scheme is incorporated owing to its good performance of exploiting the instance and label relationship. Meanwhile, since conventional one-to-one graph matching algorithm does not satisfy the constraint of PML problem that multiple instances may correspond to multiple labels, we extend the traditional probabilistic graph matching algorithm from one-to-one constraint to many-to-many constraint, and make the proposed framework to accommodate to the PML problem. Moreover, to improve the performance of predictive model, both the minimum error reconstruction and k-nearest-neighbor weight voting scheme are employed to assign more accurate labels for unseen instances. Extensive experiments on various data sets demonstrate the superiority of our proposed method.

Partial label learning (PLL) aims to learn from the data where each training instance is associated with a set of candidate labels, among which only one is correct. Most existing methods deal with this type of problem by either treating each candidate label equally or identifying the ground-truth label iteratively. In this article, we propose a novel PLL approach named HERA, which simultaneously incorporates the HeterogEneous Loss and the SpaRse and Low-rAnk procedure to estimate the labeling confidence for each instance while training the desired model. Specifically, the heterogeneous loss integrates the strengths of both the pairwise ranking loss and the pointwise reconstruction loss to provide informative label ranking and reconstruction information for label identification, whereas the embedded sparse and low-rank scheme constrains the sparsity of ground-truth label matrix and the low rank of noise label matrix to explore the global label relevance among the whole training data, for improving the learning model. Comprehensive ablation study demonstrates the effectiveness of our employed heterogeneous loss, and extensive experiments on both artificial and real-world datasets demonstrate that our method achieves superior or comparable performance against state-of-the-art methods.

Although the performance of automatically generating high-resolution realistic images from text descriptions has been significantly boosted, many challenging issues in image synthesis have not been fully investigated, due to shapes variations, viewpoint changes, pose changes, and the relations of multiple objects. In this article, we propose a novel end-to-end approach for text-to-image synthesis with spatial constraints by mining object spatial location and shape information. Instead of learning a hierarchical mapping from text to image, our algorithm directly generates multi-object fine-grained images through the guidance of the generated semantic layouts. By fusing text semantic and spatial information into a synthesis module and jointly fine-tuning them with multi-scale semantic layouts generated, the proposed networks show impressive performance in text-to-image synthesis for complex scenes. We evaluate our method both on single-object CUB dataset and multi-object MS-COCO dataset. Comprehensive experimental results demonstrate that our method significantly outperforms the state-of-the-art approaches consistently across different evaluation metrics.

Recently, co-saliency detection, which aims to automatically discover common and salient objects appeared in several relevant images, has attracted increased interest in the computer vision community. In this article, we present a novel graph-matching based model for co-saliency detection in image pairs. A solution of graph matching is proposed to integrate the visual appearance, saliency coherence, and spatial structural continuity for detecting co-saliency collaboratively. Since the saliency and the visual similarity have been seamlessly integrated, such a joint inference schema is able to produce more accurate and reliable results. More concretely, the proposed model first computes the intra-saliency for each image by aggregating multiple saliency cues. The common and salient regions across multiple images are thus discovered via a graph matching procedure. Then, a graph reconstruction scheme is proposed to refine the intra-saliency iteratively. Compared to existing co-saliency detection methods that only utilize visual appearance cues, our proposed model can effectively exploit both visual appearance and structure information to better guide co-saliency detection. Extensive experiments on several challenging image pair databases demonstrate that our model outperforms state-of-the-art baselines significantly.