Open-world semi-supervised learning (OWSSL) extends traditional semi-supervised learning to open-world scenarios by identifying novel categories in unlabeled data, thereby enhancing the model's generalization capability. However, existing OWSSL datasets typically assume a balanced class distribution, whereas real-world applications often exhibit highly imbalanced distributions. This imbalance makes it particularly challenging to learn tail classes and discover novel categories. This paper introduces the Class-Balanced Representation and Recognition Framework (CBTM-NCD), which uses the Variational Dirichlet Process (VDP) to improve tail class features and includes a generative data balancing strategy.Additionally, CBTM-NCD adopts a two-stage optimization strategy to identify novel category samples, effectively tackling three major challenges prevalent in open-world long-tailed scenarios in open-world long-tailed distributions: insufficient feature representation of tail classes, difficulty in discovering unknown categories, and class distribution imbalance.To enhance transparency and reproducibility, the code is available at https://github.com/wuzelei123/CBTM-NCD.

Neural Radiance Fields (NeRF) have advanced 3D reconstruction by learning implicit representations of scenes from multi-view images, yet their effectiveness is limited in environments with scattering medium. Existing methods that incorporate scattering models into NeRF frameworks face issues with slow training speeds and high memory demands. This paper presents DehazeGS, a novel haze removal and reconstruction method based on 3D Gaussian Splatting (3DGS). Our approach integrates the Koschmieder scattering model into the 3DGS framework, enabling effective separation of objects and scattering medium. This method leverages a point-based representation to achieve high-quality scene reconstruction while significantly reducing computational and memory overhead. Experimental results on both synthetic and real datasets demonstrate that our method outperforms existing approaches in terms of dehazing quality and reconstruction performance, effectively synthesizing clear images from foggy scenes. Our findings suggest that integrating scattering models with 3DGS offers a promising solution for applications in adverse weather conditions.

The generation of 3D meshes is critical in numerous applications, evidenced by the growing popularity and attention towards interactive generative models. Although diffusion models currently stand out as powerful interactive generative models, they are confined to the 2D domain. Performing direct diffusion and denoising on complex 3D meshes with dense vertices and faces is impractical, time-consuming, and resource-intensive. In this work, we discretize the 3D space and incorporate the intricate 3D mesh topology within the Truncated-Signed Distance Fields (T-SDFs) of each discrete cell vertex and propose an efficient discrete index graph diffusion model for T-SDFs. We further divide T-SDFs into multiple local shapes and encode the complete object as a discretized 3D grid based on codebook indices, with each index labeled for its position to preserve its discretization while reducing the input dimensionality. A graph neural network is trained on these latent spaces to jointly denoise the diffusion process on continuous coordinates and discrete codebook indices to incorporate local and global information. As we delete the most frequently repeated codebook indices in the 3D grid, the input size of the diffusion model becomes variable. We employ diverse conditional embeddings from task-specific autoencoders to estimate the quantity of codebook indices in various 3D grids and achieve interactive conditional synthesis by utilizing classifier-free guidance to sample from diverse normal distributions. Our model exhibits exceptional generative performance, supported by experimental results showcasing its effectiveness in various generative tasks, including shape completion, single-view 3D generation, and text-driven generation.

Spectral clustering is widely used in the field of unsupervised learning and has been successfully applied in various data analysis tasks. However, in practical scenarios, it’s common to get few labeled data at low cost, resulting limited labeled data in few categories, while the rest are completely unlabeled. Relying solely on the unsupervised clustering framework can’t leverage the existing supervised information. Using a supervised meta-learning paradigm alone poses challenges when dealing with the majority categories of completely unlabeled data. Therefore, we propose a few-shot contrastive spectral clustering combined with meta learning and neighbor mining framework (FCSMN). First we leverage few shot labeled information to obtain a meta-learning feature embedding with strong generalization capabilities, and acquire semantic feature spaces using an unsupervised contrastive learning model. Then, we mine neighbors in these two feature spaces separately and align them. Next, we jointly reinforce intra-cluster compactness and inter-cluster separability at both instance and cluster level. To satisfies the conditions of spectral clustering, we impose orthogonality constraints at the last layer. Experiments on three datasets demonstrate the effectiveness our proposed method.

Abstract Deep spectral clustering techniques are considered one of the most efficient clustering algorithms in data mining field. The similarity between instances and the disparity among classes are two critical factors in clustering fields. However, most current deep spectral clustering approaches do not sufficiently take them both into consideration. To tackle the above issue, we propose Semantic Spectral clustering with Contrastive learning and Neighbor mining (SSCN) framework, which performs instance-level pulling and cluster-level pushing cooperatively. Specifically, we obtain the semantic feature embedding using an unsupervised contrastive learning model. Next, we obtain the nearest neighbors partially and globally, and the neighbors along with data augmentation information enhance their effectiveness collaboratively on the instance level as well as the cluster level. The spectral constraint is applied by orthogonal layers to satisfy conventional spectral clustering. Extensive experiments demonstrate the superiority of our proposed frame of spectral clustering.

The graph convolutional network (GCN), as a powerful tool in graph data processing, is widely exploited in many machine learning and computer vision tasks. However, existing GCNs usually assume that the network has fixed outputs, which is usually contrary to the real-world class number being unknown and incremental, leading to an open set classification problem in which the finite training dataset cannot contain all labels in the infinite testing data. To overcome these issues, a novel Bayesian model is proposed, in which we couple GCN and a deep generative clustering model in a unified framework. In our model, the GCN model is used to detect the known classes, the deep generative clustering model is designed to generate the novel classes, and a two-level label generative process is constructed to extend the finite GCN outputs to infinity and fuse the label generated by the GCN model and the deep generative model. Although posterior inference is difficult, our model leads to an efficient variational inference-based optimization method. Experiments on various datasets validate our theoretical analysis and demonstrate that our model can achieve state-of-the-art performance. Our source code has been released on the website.

Deep spectral clustering is a popular and efficient algorithm in unsupervised learning. However, deep spectral clustering methods are organized into three separate steps: affinity matrix learning, spectral embedding learning, and K-means clustering on spectral embedding. In this case, although each step can achieve its own performance, it is still difficult to obtain robust clustering results. In this letter, we propose a one-step adaptive spectral clustering network to overcome the aforementioned shortcomings. The network embeds the three parts of affinity matrix learning, spectral embedding learning, and indicator learning into a unified framework. The affinity matrix is adaptively adjusted by spectral embedding in a deep subspace. We introduce spectral rotation to discretize spectral embedding, which makes the spectral embedding and indicator be learned simultaneously to improve clustering quality. Each part of the model can be iteratively updated based on other parts to optimize the clustering results. Experimental results on four real datasets show the effectiveness of our method on the ACC and NMI clustering evaluation metrics. In particular, our method achieves an NMI of 0.932 and an ACC of 0.973 on the MNIST dataset, a decent performance boost compared to the best baseline.

Sparse subspace clustering, as one of the most effective subspace clustering method, is widely studied in the data processing realm. However, conventional sparse subspace clustering methods are organized with two separated steps, feature learning and indicator learning. This makes the algorithm suffer the difficulties that: (1) representation and cluster indicator learning cannot affect each other; (2) cluster number and sparse penalty coefficient should be specified a priori; (3) sparse subspace clustering method is designed for the linear subspace data and cannot be exploited in the general clustering task. In this paper, a novel sparse clustering method is proposed, in which we extend the conventional algebraic sparse subspace clustering approach to a Bayesian framework. Then, cluster number estimation and low rank constraint are coupled via a Dirichlet process parameter generation process, in which the rank are no more required to be low but can be generated with a suitable value. Finally, Generative Adversarial Network (GAN) is incorporated into the Bayesian sparse model, which extends the subspace clustering method to a normal clustering model. Experiments on different real world datasets validate our theory analysis and demonstrate the effective of the proposed algorithm.

Applying convolution methods to domains that lack regular underlying structures is a challenging task for 3D vision. Existing methods require the manual design of feature representations suitable for the task or full-voxel-level analysis, which is memory intensive. In this paper, we propose a novel feature extraction method to facilitate 3D nonrigid shape analysis. Our approach, called 3D-MConv, extends convolution operations from regular grids to irregular mesh sets by parametrizing a series of convolutional templates and adopts a novel local perspective to ensure that the algorithm is more invariant against global isometric deformation and articulation. We carefully design the convolutional template as a polynomial function that flexibly represents the local shape. An unsupervised learning method is adopted to learn the convolutional template function. By using the convolution operation and the movement of the template on the model surface, we can obtain the distribution of the typical template shapes. We combine this distribution feature with the spatial co-occurrence information of typical template shapes modelled by Markov chains to form a high-level descriptor of a 3D model. The support vector machine method is used to classify the nonrigid 3D objects. Experiments on SHREC10 and SHREC15 demonstrate that 3D-MConv achieves state-of-the-art accuracy on standard benchmarks.

The principle of generative adversarial net is to fit the given data distribution by combining a generative model and discriminative model. There are two major challenges to conventional systems – they are difficult to train and they easily fall into ‘mode collapse’. To improve it, this study describes a novel network structure with dual generators. A ‘cooperation’ mechanism is introduced to help the generators work together. During training, generators not only learn from discriminative feedback but also from each other (like a study group). Compared with a single‐generator network, a dual‐generator network could capture many more ‘modes’ and eventually reduce the impact of ‘mode collapse.’ Dual networks also require extra computational resources. However, our experiment shows that even with network parameters of similar size, dual networks still achieved better results. Additionally, a dual‐generator structure could be extended to multiple generators. The proposed network structure is also very robust and flexible. It can be adapted to various application scenarios, such as high‐resolution image generation, domain adaptation and 3D model generation. The experimental results showed that with the same computing resources, multiple generators can generate better quality synthetic data, including 2D images, 3D objects, style transferring etc.

Spectral clustering is a popular tool in many unsupervised computer vision and machine learning tasks. Recently, due to the encouraging performance of deep neural networks, many conventional spectral clustering methods have been extended to the deep framework. Although these deep spectral clustering methods are quite powerful and effective, learning the cluster number from data is still a challenge. In this paper, we aim to tackle this problem by integrating the spectral clustering, generative adversarial network and low rank model within a unified Bayesian framework. First, we adapt the low rank method to the cluster number estimation problem. Then, an adversarial-learning-based deep clustering method is proposed and incorporated. When introducing the spectral clustering method into our model clustering procedure, a hidden space structure preservation term is proposed. Via a Bayesian framework, the structure preservation term is embedded into the generative process, which can then be used to deduce a spectral clustering in the optimization procedure. Finally, we derive a variational-inference-based method and embed it into the network optimization and learning procedure. Experiments on different datasets prove that our model has the cluster number estimation capability and show that our method can outperform many similar graph clustering methods.

Being a widely studied model in machine learning and multimedia community, Deep Neural Network (DNN) has achieved an encouraging success in various applications. However, conventional DNN suffers the difficulty when handling the open set learning problem, in which the true class number is unknown, and the predication label in the testing dataset usually has unseen classes which are not contained in the training set. In this paper, we aim to tackle this problem by unifying deep neural network and Dirichlet process mixture model. Firstly, to learn the deep feature and enable the incorporation of DNN and the Bayesian nonparametric model, we extend deep metric learning to a semi-supervised framework. Secondly, with the learned deep feature, we construct our open set classification method by expanding the Dirichlet process mixture model to a semi-supervised framework. To infer our semi-supervised Bayesian model, the corresponding variational inference algorithm has also been derived. Experiment on synthetic and real world datasets validates our theory analysis and demonstrates the state-of-the-art performance.

Multimanifold clustering separates data points approximately lying on a union of submanifolds into several clusters. In this paper, we propose a new nonparametric Bayesian model to handle the manifold data structure. In our framework, we first model the manifold mapping function between Euclidean space and topological space by applying a deep neural network, and then construct the corresponding generation process of multiple manifold data. To solve the posterior approximation problem, in the optimization procedure, we apply a variational auto-encoder-based optimization algorithm. Especially, as the manifold algorithm has poor performance on the real dataset where nonmanifold and manifold clusters are appearing simultaneously, we expand our proposed manifold algorithm by integrating it with the original Dirichlet process mixture model. Experimental results have been carried out to demonstrate the state-of-the-art clustering performance.

Shape description and feature detection are fundamental problems in computer graphics and geometric modeling. Among many existing techniques, those based on geodesic distance have proven effective in providing intrinsic and discriminative shape descriptors. In this article we introduce a new intrinsic function for a three-dimensional (3D) shape and use it for shape description and geometric feature detection. Specifically, we introduce the intrinsic girth function (IGF) defined on a 2D closed surface. For a point p on the surface, the value of the IGF at p is the length of the shortest nonzero geodesic path starting and ending at p . The IGF is invariant under isometry, insensitive to mesh tessellations, and robust to surface noise. We propose a fast method for computing the IGF and discuss its applications to shape retrieval and detecting tips, tubes, and plates that are constituent parts of 3D objects.