We propose a transformation diffusion model for point cloud registration to balance precision and efficiency. Our method formulates point cloud registration as a denoising diffusion process from noisy transformation to object transformation, which is represented by quaternion and translation. Specifically, in training stage, object transformation diffuses from ground-truth transformation to random distribution, and the model learns to reverse this noising process. In sampling stage, the model refines randomly generated transformation to the optimal transformation in a progressive way. We derive the variational bound in closed form for training and provide instantiation of the model. Our diffusion model maps transformation into latent space, and splits the transformation into two components (rotation and translation) based on the fact that they belong to different solution spaces. In addition, our work provides the following crucial findings: (i) Point cloud registration, one of the representative discriminative tasks, can be solved by a generative way and mapped into latent space to obtain new unified probabilistic formulation. (ii) Our model, Transformation Diffusion Model (TDM) can be a plug-and-play agent for point cloud registration, making our method applicable to different deep registration networks. Experimental results on synthetic and real-world datasets demonstrate that, in correspondence-free and correspondence-based scenarios, TDM can both achieve exceeding 60% performance improvements and higher efficiency simultaneously.

Recommender systems play crucial roles in addressing the issue of information overload, but traditional centralized storage in recommendation poses significant privacy concerns. In recent years, federated learning has been successfully introduced into a recommendation, while these algorithms still encounter several challenges. First, real-world recommendation scenarios often suffer from sparse data, making it difficult for models to learn reliable representations. Second, data heterogeneity necessitates the design of personalized models to enhance recommendation performance. To address these challenges, we propose a federated recommendation approach based on graph neural networks, named federated personalized contrastive learning for recommendation. On the client side, we propose a contrastive learning approach to enhance the embedding quality of nodes (users or items) by maximizing positive similarities. Specifically, we formulate the concept of structural neighbors based on the graph structure and devise a contrastive learning objective. We treat nodes and their structural neighbors as positive pairs to better learn node representations. On the server side, we group users based on the learned representations and compute cluster-level federated models and a global model. Each user learns a personalized model by combining these two models. Extensive experiments on five real-world datasets demonstrate that the proposed algorithm outperforms existing methods in terms of performance.

Unsupervised point cloud registration is crucial in 3D computer vision. However, most unsupervised methods struggle to construct effective optimization objectives and reliable unsupervised signals to enhance the performance of the model. To address these issues, with the observation of the significant alignment between the registration process and the Markov Decision Process (MDP), we model point cloud registration as MDP, which can provide more reliable unsupervised signals through the reward. We propose a colored noise based cross-entropy method, which introduces colored noise into sampling process, regulating the power spectral density of the action sequence and expanding the search space, improving the registration effect. Particularly, to strengthen constraints on MDP and training in the transformation space, we utilize equivariance theory to construct transformation equivariant constraint as a new optimization objective and derive equivariant constraint solutions for optimization, providing more reliable unsupervised signals. Extensive experiments demonstrate the superior performance of our method on benchmark datasets.

The empirical studies of most existing graph neural networks (GNNs) broadly take the original node feature and adjacency relationship as single-channel input, ignoring the rich information of multiple graph channels. To circumvent this issue, the multichannel graph analysis framework has been developed to fuse graph information across channels. How to model and integrate shared (i.e., consistency) and channel-specific (i.e., complementarity) information is a key issue in multichannel graph analysis. In this article, we propose a cross-channel graph information bottleneck (CCGIB) principle to maximize the agreement for common representations and the disagreement for channel-specific representations. Under this principle, we formulate the consistency and complementarity information bottleneck (IB) objectives. To enable optimization, a viable approach involves deriving variational lower bound and variational upper bound (VarUB) of mutual information terms, subsequently focusing on optimizing these variational bounds to find the approximate solutions. However, obtaining the lower bounds of cross-channel mutual information objectives proves challenging through direct utilization of variational approximation, primarily due to the independence of the distributions. To address this challenge, we leverage the inherent property of joint distributions and subsequently derive variational bounds to effectively optimize these information objectives. Extensive experiments on graph benchmark datasets demonstrate the superior effectiveness of the proposed method.

Graph Structure Learning (GSL) has demonstrated considerable potential in the analysis of graph-unknown non-Euclidean data across a wide range of domains. However, constructing an end-to-end graph structure learning model poses a challenge due to the impediment of gradient flow caused by the nearest neighbor sampling strategy. In this paper, we construct a differential graph structure learning model by replacing the non-differentiable nearest neighbor sampling with a differentiable sampling using the reparameterization trick. Under this framework, we argue that the act of sampling nearest neighbors may not invariably be essential, particularly in instances where node features exhibit a significant degree of similarity. To alleviate this issue, the bell-shaped Gaussian Similarity (GauSim) modeling is proposed to sample non-nearest neighbors. To adaptively model the similarity, we further propose Neural Gaussian Similarity (NeuralGauSim) with learnable parameters featuring flexible sampling behaviors. In addition, we develop a scalable method by transferring the large-scale graph to the transition graph to significantly reduce the complexity. Experimental results demonstrate the effectiveness of the proposed methods.

Inliers estimation constitutes a pivotal step in partially overlapping point cloud registration. Existing methods broadly obey coordinate-based scheme, where inlier con-fidence is scored through simply capturing coordinate differences in the context. However, this scheme results in massive inlier misinterpretation readily, consequently affecting the registration performance. In this paper, we explore to extend a new definition called inlier confidence calibration (ICC) to alleviate the above issues. Firstly, we provide finely initial correspondences for ICC in order to generate high quality reference point cloud copy corresponding to the source point cloud. In particular, we develop a soft assignment matrix optimization theorem that offers faster speed and greater precision compared to Sinkhorn. Benefiting from the high quality reference copy, we argue the neighborhood patch formed by inlier and its neighborhood should have consistency between source point cloud and its reference copy. Based on this insight, we construct transformation-invariant geometric constraints and capture geometric structure consistency to calibrate inlier confidence for estimated correspondences between source point cloud and its reference copy. Finally, transformation is further calculated by the weighted SVD algorithm with the calibrated inlier confidence. Our model is trained in an unsupervised manner, and extensive experiments on synthetic and real-world datasets illustrate the effectiveness of the proposed method.

Multi-instance point cloud registration is the problem of estimating multiple rigid transformations between two point clouds. Existing solutions rely on global spatial consistency of ambiguity and the time-consuming clustering of highdimensional correspondence features, making it difficult to handle registration scenarios where multiple instances overlap. To address these problems, we propose a maximal clique based multiinstance point cloud registration framework called PointMC. The key idea is to search for maximal cliques on the correspondence compatibility graph to estimate multiple transformations, and cluster these transformations into clusters corresponding to different instances to efficiently and accurately estimate all poses. PointMC leverages a correspondence embedding module that relies on local spatial consistency to effectively eliminate outliers, and the extracted discriminative features empower the network to circumvent missed pose detection in scenarios involving multiple overlapping instances. We conduct comprehensive experiments on both synthetic and real-world datasets, and the results show that the proposed PointMC yields remarkable performance improvements.

Extracting discriminative representations is the key step for correspondence-free point cloud registration. The extracted representations require to be discriminative to transformation, which demands representations to reduce the influence of redundant information irrelevant to transformation. However, recently proposed methods ignore this crucial property, resulting in limited ability to represent point cloud. In addition, researching correspondence-free point cloud registration has stagnated in recent years. In this paper, we try to relieve features redundancy issue for correspondence-free point cloud registration from a new perspective. Specifically, our method comprises two stages: feature extraction stage and rigid body transformation stage. In feature extraction stage, we aim to maximize multi-hierarchical mutual information between different hierarchical features, which can provide discriminative and less redundancy representations to regress transformation parameters for next stage. In rigid body transformation stage, we utilize dual quaternion to estimate transformation parameters, which combines rotation and translation simultaneously within a unified framework and obtains a compact representations for rigid transformation. The proposed model is trained in an unsupervised manner on the ModelNet40 dataset. The experimental results illustrate that our method achieves higher accuracy and robustness compared with existing correspondence-free methods.

Learning effective representations from unlabeled data is a challenging task for point cloud understanding. As the human visual system can map concepts learned from 2D images to the 3D world, and inspired by recent multimodal research, we introduce data from point cloud modality and image modality for joint learning. Based on the properties of point clouds and images, we propose CrossNet, a comprehensive intra- and cross-modal contrastive learning method that learns 3D point cloud representations. The proposed method achieves 3D-3D and 3D-2D correspondences of objectives by maximizing the consistency of point clouds and their augmented versions, and with the corresponding rendered images in invariant space. We further distinguish the rendered images into RGB and grayscale images to extract color and geometric features, respectively. These training objectives combine feature correspondences between modalities to combine rich learning signals from point clouds and images. Our CrossNet is simple: we add a feature extraction module and a projection head module to the point cloud and image branches, respectively, to train the backbone network in a self-supervised manner. After the network is pretrained, only the point cloud feature extraction module is required for fine-tuning and directly predicting results for downstream tasks. Our experiments on multiple benchmarks demonstrate improved point cloud classification and segmentation results, and the learned representations can be generalized across domains.

Three-dimensional point cloud registration is an important field in computer vision. Recently, due to the increasingly complex scenes and incomplete observations, many partial-overlap registration methods based on overlap estimation have been proposed. These methods heavily rely on the extracted overlapping regions with their performances greatly degraded when the overlapping region extraction underperforms. To solve this problem, we propose a partial-to-partial registration network (RORNet) to find reliable overlapping representations from the partially overlapping point clouds and use these representations for registration. The idea is to select a small number of key points called reliable overlapping representations from the estimated overlapping points, reducing the side effect of overlap estimation errors on registration. Although it may filter out some inliers, the inclusion of outliers has a much bigger influence than the omission of inliers on the registration task. The RORNet is composed of overlapping points' estimation module and representations' generation module. Different from the previous methods of direct registration after extraction of overlapping areas, RORNet adds the step of extracting reliable representations before registration, where the proposed similarity matrix downsampling method is used to filter out the points with low similarity and retain reliable representations, and thus reduce the side effects of overlap estimation errors on the registration. Besides, compared with previous similarity-based and score-based overlap estimation methods, we use the dual-branch structure to combine the benefits of both, which is less sensitive to noise. We perform overlap estimation experiments and registration experiments on the ModelNet40 dataset, outdoor large scene dataset KITTI, and natural data Stanford Bunny dataset. The experimental results demonstrate that our method is superior to other partial registration methods. Our code is available at https://github.com/superYuezhang/RORNet.

We explore the effect of geometric structure descriptors on extracting reliable correspondences and obtaining accurate registration for point cloud registration. The point cloud registration task involves the estimation of rigid transformation motion in unorganized point cloud, hence it is crucial to capture the contextual features of the geometric structure in point cloud. Recent coordinates-only methods ignore numerous geometric information in the point cloud which weaken ability to express the global context. We propose Enhanced Geometric Structure Transformer to learn enhanced contextual features of the geometric structure in point cloud and model the structure consistency between point clouds for extracting reliable correspondences, which encodes three explicit enhanced geometric structures and provides significant cues for point cloud registration. More importantly, we report empirical results that Enhanced Geometric Structure Transformer can learn meaningful geometric structure features using none of the following: (i) explicit positional embeddings, (ii) additional feature exchange module such as cross-attention, which can simplify network structure compared with plain Transformer. Extensive experiments on the synthetic dataset and real-world datasets illustrate that our method can achieve competitive results.

Point cloud registration is a key problem in the application of computer vision to robotics, autopilot and other fields. However, because the object is partially covered up or the resolution of 3D scanners is different, point clouds collected by the same sense may be inconsistent and even incomplete. Inspired by the recently proposed learning-based approaches, we propose Inliers Estimation Network (INENet) which includes a self-designed threshold prediction network and a probability estimation network with adaptive similarity mutual attention to help to find the overlapping area of the point clouds. In order to solve the above problems, we divide the partially overlapping point cloud registration task into two sub-tasks: overlapping areas detection and registration. The threshold prediction network can automatically calculate the threshold according to the input point clouds, and then the probability estimation network estimates the overlapping points by using threshold. The advantages of the proposed approach include: (1) threshold prediction network avoids bias and the complexity of manually adjusting the threshold. (2) Probability estimation network with similarity matrix can deeply fuse the information between a pair of point clouds, which is helpful to improve the accuracy. (3) INENet can be easily integrated into other overlapping region sensitive algorithms and without adjusting parameters. We conduct experiments on the ModelNet40, S3DIS and 3DMatch data sets. Specifically, the rotation error of the registration algorithm integrated with INENet is improved by at least 25% compared with direct partial overlap registration, our method improves the$F_{1} $score by 5% and has better anti-noise ability compared with the existing overlap detection methods, showing the effectiveness of the proposed method.

Recently, maximizing mutual information has emerged as a powerful tool for unsupervised graph representation learning. Existing methods are typically effective in capturing graph information from the topology view but consistently ignore the node feature view. To circumvent this problem, we propose a novel method by exploiting mutual information maximization across feature and topology views. Specifically, we first construct the feature graph to capture the underlying structure of nodes in feature spaces by measuring the distance between pairs of nodes. Then we use a cross-view representation learning module to capture both local and global information content across feature and topology views on graphs. To model the information shared by the feature and topology spaces, we develop a common representation learning module by using mutual information maximization and reconstruction loss minimization. Here, minimizing reconstruction loss forces the model to learn the shared information of feature and topology spaces. To explicitly encourage diversity between graph representations from the same view, we also introduce a disagreement regularization to enlarge the distance between representations from the same view. Experiments on synthetic and real-world datasets demonstrate the effectiveness of integrating feature and topology views. In particular, compared with the previous supervised methods, the proposed method achieves comparable or even better performance under the unsupervised representation and linear evaluation protocol.

Graph Neural Network (GNN) is capable of applying deep neural networks to graph domains. Recently, Message Passing Neural Networks (MPNNs) have been proposed to generalize several existing graph neural networks into a unified framework. For graph representation learning, MPNNs first generate discriminative node representations using the message passing function and then read from the node representation space to generate a graph representation using the readout function. In this paper, we analyze the representation capacity of the MPNNs for aggregating graph information and observe that the existing approaches ignore the self-loop for graph representation learning, leading to limited representation capacity. To alleviate this issue, we introduce a simple yet effective propagation enhanced extension, Self-Connected Neural Message Passing (SC-NMP), which aggregates the node representations of the current step and the graph representation of the previous step. To further improve the information flow, we also propose a Densely Self-Connected Neural Message Passing (DSC-NMP) that connects each layer to every other layer in a feed-forward fashion. Both proposed architectures are applied at each layer and the graph representation can then be used as input into all subsequent layers. Remarkably, combining these two architectures with existing GNN variants can improve these models’ performance for graph representation learning. Extensive experiments on various benchmark datasets strongly demonstrate the effectiveness, leading to superior performance for graph classification and regression tasks.

As a unified framework for graph neural networks, message passing-based neural network (MPNN) has attracted a lot of research interest and has been shown successfully in a number of domains in recent years. However, because of over-smoothing and vanishing gradients, deep MPNNs are still difficult to train. To alleviate these issues, we first introduce a deep hierarchical layer aggregation (DHLA) strategy, which utilizes a block-based layer aggregation to aggregate representations from different layers and transfers the output of the previous block to the subsequent block, so that deeper MPNNs can be easily trained. Additionally, to stabilize the training process, we also develop a novel normalization strategy, neighbor normalization (NeighborNorm), which normalizes the neighbor of each node to further address the training issue in deep MPNNs. Our analysis reveals that NeighborNorm can smooth the gradient of the loss function, i.e., adding NeighborNorm makes the optimization landscape much easier to navigate. Experimental results on two typical graph pattern-recognition tasks, including node classification and graph classification, demonstrate the necessity and effectiveness of the proposed strategies for graph message-passing neural networks.

Graph classification is a practical problem in many different domains including bioinformatics, chemoinformatics, social network analysis, and etc. For the graph classification task, the existing graph neural network approaches usually generate graph features using graph pooling at each step. However, this strategy of pooling only at the current step ignores the impact of self-loop. To eliminate this limitation, we propose a novel self-loop graph pooling strategy that can utilize the node information of the current step and the graph representation information of the previous step to generate an effective representation for the graph classification task. Further to measure the importance of self-loop, we also develop a gated approach, gated graph pooling with self-loop, that utilizes the simple fusion gate to enhance the representation capacity of the model. We evaluate our model on common benchmark datasets and experimental results have demonstrated the superior performance improvement on predictive accuracy.

Effectively highlighting multiple changes in the earth surface from multi-temporal remote sensing images is a meaningful but challenging task. In order to reduce costs and ensure the performance, it is advisable to employ a semisupervised strategy to achieve this goal. As a discriminative joint classification task, semisupervised change detection aims to extract useful and discriminative features from a large amount of unlabeled data in addition to limited labeled samples. The discriminator of a well-trained generative adversarial network (GAN) is just right for this. Therefore, in this letter, we proposed a semisupervised GAN-based multiple change detection framework for multi-spectral images. First, the GAN is trained by all data without any prior information. Then, we combine two identical trained discriminators to construct a dual-pipeline joint classifier. Finally, the classifier is fine-tuned by a very small amount of labeled data to detect multiple changes. The superior performance of the proposed model over both real multi-spectral data sets demonstrates its robustness and effectiveness.

This paper presents a comparative analysis of segmentation and non-segmentation based techniques for cursive handwritten word recognition. In our segmentation based technique, every word is segmented into characters, the chain code features are extracted from segmented characters, the features are fed to neural network classifier and finally the words are constructed using a string compare algorithm. In our non-segmentation based technique, the chain code features are extracted directly from words and the words are fed to a neural network classifier to classify them into word classes. To make a fair comparison, a CEDAR benchmark database is used, and the parameters such as the number of words, thresholding, resizing, feature extraction techniques, etc. are kept same for both the techniques. The experimental results and analysis show that the non-segmentation technique achieves higher recognition rate than the segmentation based technique.