Efficiency is crucial for primitive segmentation in industrial applications. Previous state-of-the-art (SOTA) methods suffer from low efficiency due to reliance on time-consuming feature clustering. To circumvent the need for slowly grouping points in a high-dimensional space, we propose a fast framework named PrimitiveGroup (PG), which makes full use of spatial relations and various feature consistency to group points efficiently in the 3-D space. Moreover, to improve the accuracy of point grouping in the 3-D space, we also introduce an adaptive long-range offset prediction module which expands the neighborhood perception range and adaptively focuses on those neighborhoods exhibiting higher semantic and instance correlation. A hybrid consistency aggregation that considers not only spatial distance and semantic constraints but also other geometric consistency of each point is proposed to decompose mixed points belonging to primitives with overlapping or adjacent centroids. Experimental results on the ABCParts and the ANSI datasets show that PG not only achieves competitive performance compared to recent SOTA methods but also operates as the fastest deep learning method in primitive segmentation, 22 times faster than the existing fastest method. Meanwhile, PG achieves promising robustness on noisy point clouds and industrial real scans.

Spiking neural networks (SNNs), inspired by the inherent spiking computation paradigm of the biological neural systems, have exhibited superior energy efficiency in 2D classification tasks over traditional artificial neural networks (ANNs). However, the regression potential of SNNs has not been well explored, especially in 3D point cloud processing. In this paper, we propose noise-injected spiking graph convolutional networks to leverage the full regression potential of SNNs in 3D point cloud denoising. Specifically, we first emulate the noise-injected neuronal dynamics to build noise-injected spiking neurons. On this basis, we design noise-injected spiking graph convolution for promoting disturbance-aware spiking representation learning on 3D points. Starting from the spiking graph convolution, we build two SNN-based denoising networks. One is a purely spiking graph convolutional network, which achieves low accuracy loss compared with some ANN-based alternatives, while resulting in significantly reduced energy consumption on two benchmark datasets, PU-Net and PC-Net. The other is a hybrid architecture, which integrates some ANN-based learning operations and exhibits a high performance-efficiency trade-off with only a few time steps. Our work lights up SNN’s potential for 3D point cloud denoising, injecting new perspectives of exploring the deployment on neuromorphic chips while paving the way for developing energy-efficient 3D data acquisition devices.

Extracting geometric edges from unstructured point clouds remains a significant challenge, particularly in thin-walled structures that are commonly found in everyday objects. Traditional geometric methods and recent learning-based approaches frequently struggle with these structures, as both rely heavily on sufficient contextual information from local point neighborhoods. However, 3D measurement data of thin-walled structures often lack the accurate, dense, and regular neighborhood sampling required for reliable edge extraction, resulting in degraded performance.In this work, we introduce STAR-Edge, a novel approach designed for detecting and refining edge points in thin-walled structures. Our method leverages a unique representation—the local spherical curve—to create structure-aware neighborhoods that emphasize co-planar points while reducing interference from close-by, non-co-planar surfaces. This representation is transformed into a rotation-invariant descriptor, which, combined with a lightweight multi-layer perceptron, enables robust edge point classification even in the presence of noise and sparse or irregular sampling. Besides, we also use the local spherical curve representation to estimate more precise normals and introduce an optimization function to project initially identified edge points exactly on the true edges. Experiments conducted on the ABC dataset and thin-walled structure-specific datasets demonstrate that STAR-Edge outperforms existing edge detection methods, showcasing better robustness under various challenging conditions. The source code is available at https://github.com/miraclelzk/star-edge.

The scaling law for large language models (LLMs) depicts that the path towards machine intelligence necessitates training at large scale. Thus, companies continuously build large-scale GPU clusters, and launch training jobs that span over thousands of computing nodes. However, LLM pre-training presents unique challenges due to its complex communication patterns, where GPUs exchange data in sparse yet high-volume bursts within specific groups. Inefficient resource scheduling exacerbates bandwidth contention, leading to suboptimal training performance. This paper presents Arnold, a scheduling system summarizing our experience to effectively align LLM communication patterns to data center topology at scale. In-depth characteristic study is performed to identify the impact of physical network topology to LLM pre-training jobs. Based on the insights, we develop a scheduling algorithm to effectively align communication patterns to physical network topology in data centers. Through simulation experiments, we show the effectiveness of our algorithm in reducing the maximum spread of communication groups by up to $1.67$x. In production training, our scheduling system improves the end-to-end performance by $10.6\%$ when training with more than $9600$ Hopper GPUs, a significant improvement for our training pipeline.

The training scale of large language models (LLMs) has reached tens of thousands of GPUs and is still continuously expanding, enabling faster learning of larger models. Accompanying the expansion of the resource scale is the prevalence of failures (CUDA error, NaN values, job hang, etc.), which poses significant challenges to training stability. Any large-scale LLM training infrastructure should strive for minimal training interruption, efficient fault diagnosis, and effective failure tolerance to enable highly efficient continuous training. This paper presents ByteRobust, a large-scale GPU infrastructure management system tailored for robust and stable training of LLMs. It exploits the uniqueness of LLM training process and gives top priorities to detecting and recovering failures in a routine manner. Leveraging parallelisms and characteristics of LLM training, ByteRobust enables high-capacity fault tolerance, prompt fault demarcation, and localization with an effective data-driven approach, comprehensively ensuring continuous and efficient training of LLM tasks. ByteRobust is deployed on a production GPU platform with over 200,000 GPUs and advances the state of the art in training robustness by achieving 97% ETTR for a three-month training job on 9,600 GPUs.

We present a novel correspondence-learning model for real-time registration of partially overlapping point clouds, between which the relative translation is large and hence identifying the correspondences is challenging. Our goal is to improve the feature learning for accurate correspondence establishment, which enables the promotion of registration performance significantly in terms of efficiency. This is realized by two particular designs. The first is a graph-based feature extraction module, which aggregates both inter and intra contexts of the input point clouds simultaneously for strengthening the connection between inputs. The second is a feature refinement module, which uses sparse convolutions to further widen the feature differences of dissimilar structures. The two modules reinforce each other to improve correspondence learning for robust and fast point cloud registration with low overlap. We evaluate the method on both synthetic and real-world large-scale datasets. The results in real registration tasks show that our method attains competitive registration accuracy with state-of-the-art methods, and is almost two times faster than these competing methods in some scenarios.

Physically-based-rendering (PBR) materials are crucial in modern rendering pipelines, and many studies have focused on acquiring these materials from reality or images. However, existing methods may result in non-tileable results, since the realistic inputs usually have seams. Compared to non-tileable materials, tileable PBR materials have more universal application scenarios. To address this issue, we introduce MaTi, a novel pipeline that converts non-tileable PBR materials into tileable ones with minimal distortion. MaTi rearranges material patches to align boundaries at the center of the image, and then uses a diffusion model to inpaint the seams. We use scaled gamma correction to reduce the occurrence of collapse when processing special material maps. The color correction and triangular blending are adopt to preserve the original material information. Additionally, we design a division and blending strategy to efficiently handle high resolution materials. Our experiments demonstrate that MaTi can seamlessly modify PBR materials while preserving the original information, outperforming existing synthesis methods.

Abstract Spherical harmonics are a favorable technique for 3D representation, employing a frequency‐based approach through the spherical harmonic transform (SHT). Typically, SHT is performed using equiangular sampling grids. However, these grids are non‐uniform on spherical surfaces and exhibit local anisotropy, a common limitation in existing spherical harmonic decomposition methods. This paper proposes a 3D representation method using Fibonacci Spherical Harmonics (FSH3D). We introduce a spherical Fibonacci grid (SFG), which is more uniform than equiangular grids for SHT in the frequency domain. Our method employs analytical weights for SHT on SFG, effectively assigning sampling errors to spherical harmonic degrees higher than the recovered band‐limited function. This provides a novel solution for spherical harmonic transformation on non‐equiangular grids. The key advantages of our FSH3D method include: 1) With the same number of sampling points, SFG captures more features without bias compared to equiangular grids; 2) The root mean square error of 32‐degree spherical harmonic coefficients is reduced by approximately 34.6% for SFG compared to equiangular grids; and 3) FSH3D offers more stable frequency domain representations, especially for rotating functions. FSH3D enhances the stability of frequency domain representations under rotational transformations. Its application in 3D shape reconstruction and 3D shape classification results in more accurate and robust representations. Our code is publicly available at https://github.com/Miraclelzk/Fibonacci-Spherical-Harmonics .

Current point cloud denoising (PCD) models optimize single networks, trying to make their parameters adaptive to each point in a large pool of point clouds. Such a denoising network paradigm neglects that different points are often corrupted by different levels of noise and they may convey different geometric structures. Thus, the intricacy of both noise and geometry poses side effects including remnant noise, wrongly-smoothed edges, and distorted shape after denoising. We propose PathNet, a path-selective PCD paradigm based on reinforcement learning (RL). Unlike existing efforts, PathNet enables dynamic selection of the most appropriate denoising path for each point, best moving it onto its underlying surface. We have two more contributions besides the proposed framework of path-selective PCD for the first time. First, to leverage geometry expertise and benefit from training data, we propose a noise- and geometry-aware reward function to train the routing agent in RL. Second, the routing agent and the denoising network are trained jointly to avoid under- and over-smoothing. Extensive experiments show promising improvements of PathNet over its competitors, in terms of the effectiveness for removing different levels of noise and preserving multi-scale surface geometries. Furthermore, PathNet generalizes itself more smoothly to real scans than cutting-edge models.

Fast and accurate registration of outdoor LiDAR point clouds poses a considerable challenge for their large-scale (e.g., 300 K points) and intricate (e.g., noise and outliers) distributions. In this article, we present a fast low-frequency coarse registration method for large-scale outdoor LiDAR point clouds, dubbed FlyCore. Different from existing methods, FlyCore is very fast for practical applications and bridges current refinement registration methods smoothly for their accuracy improvements. Specifically, we first construct spherical feature spaces for a pair of point clouds based on their keypoints and saliency uncertainties independently. Then, we perform harmonic decomposition on these spherical feature spaces, utilizing the low-frequency components of spherical harmonics (SHs) to implement point cloud registration. FlyCore demonstrates less sensitivity to noise and outliers compared to feature-based registration techniques. Also, FlyCore achieves exceptionally low time complexity by eliminating the need for feature matching and iterative procedures, ensuring fine alignment with only a few iterations. Experimental validations, utilizing two extensive LiDAR datasets featuring urban and natural scenarios, confirm the effectiveness and accuracy improvement of existing fine registration methods facilitated by our FlyCore.

Point cloud denoising and normal estimation are two fundamental yet dependent problems in digital geometry processing. However, both are often independently researched, leading to inconsistent geometry on 3D surfaces. To address it, we propose PN-Internet, an end-to-end Point-and-Normal Interactive Network for joint point cloud denoising and normal estimation. PN-Internet leverages the geometric dependency between point positions and normals to design two interactive graph convolution networks (GCNs): a point-to-normal network and a normal-to-point network. It adopts a coarse-to-fine learning paradigm, where two GCNs are exploited to respectively perform point cloud denoising and normal estimation. The point-to-normal network improves the quality of the normals using an MLP module, while the normal-to-point network refines the point positions using a parameter-free projection module based on the constraints from the normals. In addition, we introduce a feature-aware loss function to preserve the quality of 3D shape features. Unlike most existing methods, PN-Internet takes advantage of the geometric dependency between points and normals and benefits from training data. Our experimental results demonstrate that PN-Internet achieves geometric consistency between point cloud denoising and normal estimation. Furthermore, we show significant improvements over state-of-the-art methods.

Mesh denoising is a fundamental problem in digital geometry processing. It seeks to remove surface noise while preserving surface intrinsic signals as accurately as possible. While traditional wisdom has been built upon specialized priors to smooth surfaces, learning-based approaches are making their debut with great success in generalization and automation. In this work, we provide a comprehensive review of the advances in mesh denoising, containing both traditional geometric approaches and recent learning-based methods. First, to familiarize readers with the denoising tasks, we summarize four common issues in mesh denoising. We then provide two categorizations of the existing denoising methods. Furthermore, three important categories, including optimization-, filter-, and data-driven-based techniques, are introduced and analyzed in detail, respectively. Both qualitative and quantitative comparisons are illustrated, to demonstrate the effectiveness of the state-of-the-art denoising methods. Finally, potential directions of future work are pointed out to solve the common problems of these approaches. A mesh denoising benchmark is also built in this work, and future researchers will easily and conveniently evaluate their methods with state-of-the-art approaches. To aid reproducibility, we release our datasets and used results at https://github.com/chenhonghua/Mesh-Denoiser .

How will you repair a physical object with some missings? You may imagine its original shape from previously captured images, recover its overall (global) but coarse shape first, and then refine its local details. We are motivated to imitate the physical repair procedure to address point cloud completion. To this end, we propose a cross-modal shape-transfer dual-refinement network (termed CSDN), a coarse-to-fine paradigm with images of full-cycle participation, for quality point cloud completion. CSDN mainly consists of "shape fusion" and "dual-refinement" modules to tackle the cross-modal challenge. The first module transfers the intrinsic shape characteristics from single images to guide the geometry generation of the missing regions of point clouds, in which we propose IPAdaIN to embed the global features of both the image and the partial point cloud into completion. The second module refines the coarse output by adjusting the positions of the generated points, where the local refinement unit exploits the geometric relation between the novel and the input points by graph convolution, and the global constraint unit utilizes the input image to fine-tune the generated offset. Different from most existing approaches, CSDN not only explores the complementary information from images but also effectively exploits cross-modal data in the whole coarse-to-fine completion procedure. Experimental results indicate that CSDN performs favorably against twelve competitors on the cross-modal benchmark.

Recently, there has been a significant surge of interest in medical image generation. In this study, we developed a model known as AE-COT-GAN (autoencoder-based conditional optimal transport generative adversarial network) to generate medical images that belong to specific categories. The primary objective of our research is to address the prevalent challenges often encountered during the training of generative adversarial networks (GANs), including issues such as mode collapse and mode mixing. The training process of our model encompasses three fundamental components. First, we employ an autoencoder model to obtain a low-dimensional manifold representation of real images. Second, we apply extended semi-discrete optimal transport to map Gaussian noise distribution to the latent space distribution and obtain corresponding labels effectively. This procedure leads to the generation of new latent codes with known labels. Finally, we integrate a GAN to train the decoder further to generate medical images. To evaluate the performance of the AE-COT-GAN model, we conducted experiments on two medical image datasets, namely DermaMNIST and BloodMNIST. The model’s performance was compared with state-of-the-art generative models. Results show that the AE-COT-GAN model had excellent performance in generating medical images. Moreover, it effectively addressed the common issues associated with traditional GANs.

Convolution on 3D point clouds is widely researched yet far from perfect in geometric deep learning. The traditional wisdom of convolution characterises feature correspondences indistinguishably among 3D points, arising an intrinsic limitation of poor distinctive feature learning. In this article, we propose Adaptive Graph Convolution (AGConv) for wide applications of point cloud analysis. AGConv generates adaptive kernels for points according to their dynamically learned features. Compared with the solution of using fixed/isotropic kernels, AGConv improves the flexibility of point cloud convolutions, effectively and precisely capturing the diverse relations between points from different semantic parts. Unlike the popular attentional weight schemes, AGConv implements the adaptiveness inside the convolution operation instead of simply assigning different weights to the neighboring points. Extensive evaluations clearly show that our method outperforms state-of-the-arts of point cloud classification and segmentation on various benchmark datasets. Meanwhile, AGConv can flexibly serve more point cloud analysis approaches to boost their performance. To validate its flexibility and effectiveness, we explore AGConv-based paradigms of completion, denoising, upsampling, registration and circle extraction, which are comparable or even superior to their competitors.

Point normal, as an intrinsic geometric property of 3D objects, not only serves conventional geometric tasks such as surface consolidation and reconstruction, but also facilitates cutting-edge learning-based techniques for shape analysis and generation. In this paper, we propose a normal refinement network, called Refine-Net, to predict accurate normals for noisy point clouds. Traditional normal estimation wisdom heavily depends on priors such as surface shapes or noise distributions, while learning-based solutions settle for single types of hand-crafted features. Differently, our network is designed to refine the initial normal of each point by extracting additional information from multiple feature representations. To this end, several feature modules are developed and incorporated into Refine-Net by a novel connection module. Besides the overall network architecture of Refine-Net, we propose a new multi-scale fitting patch selection scheme for the initial normal estimation, by absorbing geometry domain knowledge. Also, Refine-Net is a generic normal estimation framework: 1) point normals obtained from other methods can be further refined, and 2) any feature module related to the surface geometric structures can be potentially integrated into the framework. Qualitative and quantitative evaluations demonstrate the clear superiority of Refine-Net over the state-of-the-arts on both synthetic and real-scanned datasets.

Existing learning-based algorithms have a certain potential in visual odometry. In this work, we propose the solution of the learning-based method, which contains the attention mechanism and pose graph optimization. We set a self-supervised network as our backbone to cope with image data and error-heavy estimation pose for pose correction. The pre-processing camera poses involved in the network can provide prior information. Combining the advantages of the abundant feature information and efficient attention mechanism, we design a geometric attention module that is sensitive to geometrical structure from images to accurately regress the rotation matrix. Then we improve the loss function with the weights of the attention module to consider the diversity of the data. Experimental results demonstrate the effectiveness and reliability of our approach on the public datasets KITTI with monocular task and stereo task. In comparison, the proposed method is superior to the existing methods in the translation component. In the self-supervised network, learning an attention mechanism can extract an effective connect relation of feature maps. We conduct ablation experiments under the self-supervised network backbone setting different strategies, and conclude that the proposed attention module is applicable to various sequences, and provide loss function improvements on the visual odometry task.

We propose a geometry-supporting dual convolutional neural network (GeoDualCNN) for both point cloud normal estimation and denoising. GeoDualCNN fuses the geometry domain knowledge that the underlying surface of a noisy point cloud is piecewisely smooth with the fact that a point normal is properly defined only when local surface smoothness is guaranteed. Centered around this insight, we define the homogeneous neighborhood (HoNe) which stays clear of surface discontinuities, and associate each HoNe with a point whose geometry and normal orientation is mostly consistent with that of HoNe. Thus, we not only obtain initial estimates of the point normals by performing PCA on HoNes, but also for the first time optimize these initial point normals by learning the mapping from two proposed geometric descriptors to the ground-truth point normals. GeoDualCNN consists of two parallel branches that remove noise using the first geometric descriptor (a homogeneous height map, which encodes the point-position information), while preserving surface features using the second geometric descriptor (a homogeneous normal map, which encodes the point-normal information). Such geometry-supporting network architectures enable our model to leverage previous geometry expertise and to benefit from training data. Experiments with noisy point clouds show that GeoDualCNN outperforms the state-of-the-art methods in terms of both noise-robustness and feature preservation.

Abstract High‐confidence overlap prediction and accurate correspondences are critical for cutting‐edge models to align paired point clouds in a partial‐to‐partial manner. However, there inherently exists uncertainty between the overlapping and non‐overlapping regions, which has always been neglected and significantly affects the registration performance. Beyond the current wisdom, we propose a novel uncertainty‐aware overlap prediction network, dubbed UTOPIC, to tackle the ambiguous overlap prediction problem; to our knowledge, this is the first to explicitly introduce overlap uncertainty to point cloud registration. Moreover, we induce the feature extractor to implicitly perceive the shape knowledge through a completion decoder, and present a geometric relation embedding for Transformer to obtain transformation‐invariant geometry‐aware feature representations. With the merits of more reliable overlap scores and more precise dense correspondences, UTOPIC can achieve stable and accurate registration results, even for the inputs with limited overlapping areas. Extensive quantitative and qualitative experiments on synthetic and real benchmarks demonstrate the superiority of our approach over state‐of‐the‐art methods.

Abstract How will you repair a physical object with large missings? You may first recover its global yet coarse shape and stepwise increase its local details. We are motivated to imitate the above physical repair procedure to address the point cloud completion task. We propose a novel stepwise point cloud completion network (SPCNet) for various 3D models with large missings. SPCNet has a hierarchical bottom‐to‐up network architecture. It fulfills shape completion in an iterative manner, which 1) first infers the global feature of the coarse result; 2) then infers the local feature with the aid of global feature; and 3) finally infers the detailed result with the help of local feature and coarse result. Beyond the wisdom of simulating the physical repair, we newly design a cycle loss to enhance the generalization and robustness of SPCNet. Extensive experiments clearly show the superiority of our SPCNet over the state‐of‐the‐art methods on 3D point clouds with large missings. Code is available at https://github.com/1127368546/SPCNet .

Abstract The intricacy of 3D surfaces often results cutting‐edge point cloud denoising (PCD) models in surface degradation including remnant noise, wrongly‐removed geometric details. Although using multi‐scale patches to encode the geometry of a point has become the common wisdom in PCD, we find that simple aggregation of extracted multi‐scale features can not adaptively utilize the appropriate scale information according to the geometric information around noisy points. It leads to surface degradation, especially for points close to edges and points on complex curved surfaces. We raise an intriguing question – if employing multi‐scale geometric perception information to guide the network to utilize multi‐scale information, can eliminate the severe surface degradation problem? To answer it, we propose a Multi‐offset Denoising Network (MODNet) customized for multi‐scale patches. First, we extract the low‐level feature of three scales patches by patch feature encoders. Second, a multi‐scale perception module is designed to embed multi‐scale geometric information for each scale feature and regress multi‐scale weights to guide a multi‐offset denoising displacement. Third, a multi‐offset decoder regresses three scale offsets, which are guided by the multi‐scale weights to predict the final displacement by weighting them adaptively. Experiments demonstrate that our method achieves new state‐of‐the‐art performance on both synthetic and real‐scanned datasets. Our code is publicly available at https://github.com/hay-001/MODNet .

Recent 3-D scanning techniques can produce various kinds of digitized 3-D data. Most of these scanned data are in a format of unstructured point clouds. Such low-level representation of 3-D data usually contains only geometric properties (point positions), while lacking higher level structure cues, for example, feature lines. Feature lines can be defined as a visually prominent characteristic of the shape, including edges, ridges, and valley lines in multiple scales, which can support a lot of downstream applications, such as shape reconstruction and analysis. We present a two-phase algorithm for extracting line-type features on point clouds. To extract both large-scale and shallow feature lines, we first define a statistical metric to detect all potential feature points while immune to the noise to some extent. Then, for correctly reconstructing the feature lines from these identified coarse feature points, we introduce an anisotropic contracting scheme to force feature points lying on the underlying real feature lines. To illustrate the reliability of our method, various experiments have been conducted on both synthetic and raw data. Both visual and quantitative comparisons show that our method is robust to noise and can correctly extract multiscale feature lines. In addition, our method is generally applicable to robotic picking.Note to Practitioners—This article was motivated by the problem of the feature line extraction for real scanned point clouds. Feature lines, as one kind of the most important structure information, depict the basic shape of the real object in our life. Extracting this kind of shape features from the unstructured point clouds can facilitate a variety of downstream practical applications, such as product design, workpiece manufacturing, and robotic grasping. Existing approaches to detect features either heavily rely on differential quantities, which are sensitive to the noise, or need an elaborately designed local descriptor but fail to recognize small-scale features. These challenges motivate us to design a new approach aiming at extracting multiscale feature lines while keeping robustness to heavy noise. The technique developed in this work can produce high-quality feature points and feature lines, which would serve as higher level structural information and facilitate many applications. Additional applications in 6-degree-of-freedom (6-DoF) pose estimation demonstrate the potential of our method for robotic picking.

Reconstruction of high-fidelity 3D objects or scenes is a fundamental research problem. Recent advances in RGB-D fusion have demonstrated the potential of producing 3D models from consumer-level RGB-D cameras. However, due to the discrete nature and limited resolution of their surface representations (e.g., point or voxel based), existing approaches suffer from the accumulation of errors in camera tracking and distortion in the reconstruction, which leads to an unsatisfactory 3D reconstruction. In this article, we present a method using on-the-fly implicits of Hermite Radial Basis Functions (HRBFs) as a continuous surface representation for camera tracking in an existing RGB-D fusion framework. Furthermore, curvature estimation and confidence evaluation are coherently derived from the inherent surface properties of the on-the-fly HRBF implicits, which are devoted to a data fusion with better quality. We argue that our continuous but on-the-fly surface representation can effectively mitigate the impact of noise with its robustness and constrain the reconstruction with inherent surface smoothness when being compared with discrete representations. Experimental results on various real-world and synthetic datasets demonstrate that our HRBF-fusion outperforms the state-of-the-art approaches in terms of tracking robustness and reconstruction accuracy.

In this article, we propose a Multi-feature Fusion VoteNet (MFFVoteNet) framework for improving the 3D object detection performance in cluttered and heavily occluded scenes. Our method takes the point cloud and the synchronized RGB image as inputs to provide object detection results in 3D space. Our detection architecture is built on VoteNet with three key designs. First, we augment the VoteNet input with point color information to enhance the difference of various instances in a scene. Next, we integrate an image feature module into the VoteNet to provide a strong object class signal that can facilitate deterministic detections in occlusion. Moreover, we propose a Projection Non-Maximum Suppression (PNMS) method in 3D object detection to eliminate redundant proposals and hence provide more accurate positioning of 3D objects. We evaluate the proposed MFFVoteNet on two challenging 3D object detection datasets, i.e., ScanNetv2 and SUN RGB-D. Extensive experiments show that our framework can effectively improve the performance of 3D object detection.

Benefiting from the excellent performance of Siamese-based trackers, huge progress on 2D visual tracking has been achieved. However, 3D visual tracking is still under-explored. Inspired by the idea of Hough voting in 3D object detection, in this paper, we propose a Multi-level Voting Siamese Network (MLVSNet) for 3D visual tracking from outdoor point cloud sequences. To deal with sparsity in outdoor 3D point clouds, we propose to perform Hough voting on multi-level features to get more vote centers and retain more useful information, instead of voting only on the fi-nal level feature as in previous methods. We also design an efficient and lightweight Target-Guided Attention (TGA) module to transfer the target information and highlight the target points in the search area. Moreover, we propose a Vote-cluster Feature Enhancement (VFE) module to exploit the relationships between different vote clusters. Extensive experiments on the 3D tracking benchmark of KITTI dataset demonstrate that our MLVSNet outperforms state-of-the-art methods with significant margins. Code will be available at https://github.com/CodeWZT/MLVSNet.

Hough voting, as has been demonstrated in VoteNet, is effective for 3D object detection, where voting is a key step. In this paper, we propose a novel VoteNet-based 3D detector with vote enhancement to improve the detection accuracy in cluttered indoor scenes. It addresses the limitations of current voting schemes, i.e., votes from neighboring objects and background have significant negative impacts. Before voting, we replace the classic MLP with the proposed Attentive MLP (AMLP) in the backbone network to get better feature description of seed points. During voting, we design a new vote attraction loss (VALoss) to enforce vote centers to locate closely and compactly to the corresponding object centers. After voting, we then devise a vote weighting module to integrate the foreground/background prediction into the vote aggregation process to enhance the capability of the original VoteNet to handle noise from background voting. The three proposed strategies all contribute to more effective voting and improved performance, resulting in a novel 3D object detector, termed VENet. Experiments show that our method outperforms state-of-the-art methods on benchmark datasets. Ablation studies demonstrate the effectiveness of the proposed components.

Abstract Guided image filtering is one of the widely used techniques in computer vision. However, it commonly leads to over‐smoothed edges and a distorted appearance when tackling intricate texture patterns and complex noise. In this paper, a window‐aware image filtering framework based on the bilateral filter guided by the local entropy is presented. The key idea of the authors' proposed approach is to design a novel guidance input and a non‐box filtering window. Specifically, using the Gaussian spatial kernel and the local entropy, a GEF that can maintain image feature details and yield a robust guidance input for BF is constructed. Meanwhile, based on an intensity‐similar strategy, the local non‐box filtering window is designed for the further preservation of edge structures. The authors' approach not only inherits the advantages of bilateral filter i.e. simplicity, parallelisation and easiness of programming, but also is more powerful than bilateral filter and its variants. In addition, the guided entropy filter and the non‐box window can also be transplanted to other local filters and can effectively improve the filtering effects. The qualitative and quantitative experimental results demonstrate that the authors' approach has good performance in image denoising, texture (or background) smoothing, edge extraction and other applications in image processing.

Tracking and mapping functions in a monocular SLAM system remain active due to their challenging nature. In this paper, we propose a novel approach to perform the accurate and robust ego-motion estimation and provide the detail-preserving reconstruction in indoor environments. More specifically, we design a new algorithm called synchronous event measurement (SEM) to create event-based difference images (EDIs) so as to highlight frame-to-frame (F2F) difference. The observation indicates that F2F difference is highly correlated with the camera's motion change. We hereby feed EDIs into a deep convolutional neural network, in order to infer ego-motion of the camera. Subsequently, based on a monocular reconstruction framework (REMODE), we devise an algorithm named event region search or briefly ERS, to reduce possibility of mismatch on the depth estimation stage. Evaluations on a variety of datasets demonstrate the satisfactory performance of our proposed method: the ego-motion estimation is more accurate than some geometric based Visual Odometry (VO) and learning based approaches. The results are robust under extreme situations, such as brightness variation and motion blur. Meanwhile, our approach can provide more precise depth map with relatively rich textural information.

We propose improving the cross-target and cross-scene generalization of visual navigation through learning an agent that is guided by conceiving the next observations it expects to see. This is achieved by learning a variational Bayesian model, called NeoNav, which generates the next expected observations (NEO) conditioned on the current observations of the agent and the target view. Our generative model is learned through optimizing a variational objective encompassing two key designs. First, the latent distribution is conditioned on current observations and the target view, leading to a model-based, target-driven navigation. Second, the latent space is modeled with a Mixture of Gaussians conditioned on the current observation and the next best action. Our use of mixture-of-posteriors prior effectively alleviates the issue of over-regularized latent space, thus significantly boosting the model generalization for new targets and in novel scenes. Moreover, the NEO generation models the forward dynamics of agent-environment interaction, which improves the quality of approximate inference and hence benefits data efficiency. We have conducted extensive evaluations on both real-world and synthetic benchmarks, and show that our model consistently outperforms the state-of-the-art models in terms of success rate, data efficiency, and generalization.

This paper looks at this intriguing question: are single images with their details lost during deraining, reversible to their artifact-free status? We propose an end-to-end detail-recovery image deraining network (termed a DRDNet) to solve the problem. Unlike existing image deraining approaches that attempt to meet the conflicting goal of simultaneously deraining and preserving details in a unified framework, we propose to view rain removal and detail recovery as two seperate tasks, so that each part could specialize rather than trade-off between two conflicting goals. Specifically, we introduce two parallel sub-networks with a comprehensive loss function which synergize to derain and recover the lost details caused by deraining. For complete rain removal, we present a rain residual network with the squeeze-and-excitation (SE) operation to remove rain streaks from the rainy images. For detail recovery, we construct a specialized detail repair network consisting of welldesigned blocks, named structure detail context aggregation block (SDCAB), to encourage the lost details to return for eliminating image degradations. Moreover, the detail recovery branch of our proposed detail repair framework is detachable and can be incorporated into existing deraining methods to boost their performances. DRD-Net has been validated on several well-known benchmark datasets in terms of deraining robustness and detail accuracy. Comparisons show clear visual and numerical improvements of our method over the state-of-the-arts.

In this paper, we address the 3D object detection task by capturing multi-level contextual information with the self-attention mechanism and multi-scale feature fusion. Most existing 3D object detection methods recognize objects individually, without giving any consideration on contextual information between these objects. Comparatively, we propose Multi-Level Context VoteNet (MLCVNet) to recognize 3D objects correlatively, building on the state-of-the-art VoteNet. We introduce three context modules into the voting and classifying stages of VoteNet to encode contextual information at different levels. Specifically, a Patch-to-Patch Context (PPC) module is employed to capture contextual information between the point patches, before voting for their corresponding object centroid points. Subsequently, an Object-to-Object Context (OOC) module is incorporated before the proposal and classification stage, to capture the contextual information between object candidates. Finally, a Global Scene Context (GSC) module is designed to learn the global scene context. We demonstrate these by capturing contextual information at patch, object and scene levels. Our method is an effective way to promote detection accuracy, achieving new state-of-the-art detection performance on challenging 3D object detection datasets, i.e., SUN RGBD and ScanNet. We also release our code at https://github.com/NUAAXQ/MLCVNet.

In this paper, we propose a normal estimation method for unstructured point cloud. We observe that geometric estimators commonly focus more on feature preservation but are hard to tune parameters and sensitive to noise, while learning-based approaches pursue an overall normal estimation accuracy but cannot well handle challenging regions such as surface edges. This paper presents a novel normal estimation method, under the co-support of geometric estimator and deep learning. To lowering the learning difficulty, we first propose to compute a suboptimal initial normal at each point by searching for a best fitting patch. Based on the computed normal field, we design a normal-based height map network (NH-Net) to fine-tune the suboptimal normals. Qualitative and quantitative evaluations demonstrate the clear improvements of our results over both traditional methods and learning-based methods, in terms of estimation accuracy and feature recovery.

Abstract We present a near‐lighting photometric stereo (NL‐PS) system to produce digital bas‐reliefs from a physical object (set) directly. Unlike both the 2D image and 3D model‐based modelling methods that require complicated interactions and transformations, the technique using NL‐PS is easy to use with cost‐effective hardware, providing users with a trade‐off between abstract and representation when creating bas‐reliefs. Our algorithm consists of two steps: normal map acquisition and constrained 3D reconstruction. First, we introduce a lighting model, named the quasi‐point lighting model (QPLM), and provide a two‐step calibration solution in our NL‐PS system to generate a dense normal map. Second, we filter the normal map into a detail layer and a structure layer, and formulate detail‐ or structure‐preserving bas‐relief modelling as a constrained surface reconstruction problem of solving a sparse linear system. The main contribution is a WYSIWYG (i.e. what you see is what you get) way of building new solvers that produces multi‐style bas‐reliefs with their geometric structures and/or details preserved. The performance of our approach is experimentally validated via comparisons with the state‐of‐the‐art methods.

Point cloud is the primary source from 3D scanners and depth cameras. It usually contains more raw geometric features, as well as higher levels of noise than the reconstructed mesh. Although many mesh denoising methods have proven to be effective in noise removal, they hardly work well on noisy point clouds. We propose a new multi-patch collaborative method for point cloud denoising, which is solved as a low-rank matrix recovery problem. Unlike the traditional single-patch based denoising approaches, our approach is inspired by the geometric statistics which indicate that a number of surface patches sharing approximate geometric properties always exist within a 3D model. Based on this observation, we define a rotation-invariant height-map patch (HMP) for each point by robust Bi-PCA encoding bilaterally filtered normal information, and group its non-local similar patches together. Within each group, all patches are geometrically similar, while suffering from noise. We pack the height maps of each group into an HMP matrix, whose initial rank is high, but can be significantly reduced. We design an improved low-rank recovery model, by imposing a graph constraint to filter noise. Experiments on synthetic and raw datasets demonstrate that our method outperforms state-of-the-art methods in both noise removal and feature preservation.

We propose a new method for modeling the indoor scene from a single color image. With our system, the user only needs to drag a few semantic bounding boxes surrounding the objects of interest. Our system then automatically finds the most similar 3D models from the ShapeNet model repository and aligns them with the corresponding objects of interest. To achieve this, each 3D model is represented as a group of view-dependent representations generated from a set of synthesized views. We iteratively conduct object segmentation and 3D model retrieval, based on the observation that good segmentation of the objects of interest can significantly improve the accuracy of model retrieval and make it robust to cluttered background and occlusions, and in turn, the retrieved 3D models can be used to assist with object segmentation. Segmentation of all objects of interest is achieved simultaneously under a unified multi-labeling framework which fully utilizes the correspondences between the objects of interest and retrieved model images. Besides, we propose a new method to estimate the scene layout of the input image with the segmentation masks, which helps compose the resulting scene and further improves the modeling result remarkably. We verify the effectiveness of our approach through experimenting with a variety of indoor images and comparing against the relevant methods.

Video and image sources are frequently applied in the area of defect inspection in industrial community. For the recognition and classification of sewer defects, a significant number of videos and images of sewers are collected. These data are then checked by human and some traditional methods to recognize and classify the sewer defects, which is inefficient and error-prone. Previously developed features like SIFT are unable to comprehensively represent such defects. Therefore, feature representation is especially important for defect autoclassification. In this paper, we study the automatic extraction of feature representation for sewer defects via deep learning. Moreover, a complete automatic system for classifying sewer defects is proposed built on a two-level hierarchical deep convolutional neural network, which shows high performance with respect to classification accuracy. The proposed network is trained on a novel data set with over 40 000 sewer images. The system has been successfully applied in the practical production, confirming its robustness and feasibility to real-world applications. The source code and trained model are available at the project website.1

3-D pose estimation for texture-less objects remains a challenging problem. Previous works either focus on a template matching method to find the nearest template as a candidate, or construct a Hough forest, which utilizes the offset of patches to vote for the object location and pose. By contrast, in this paper, we propose a comprehensive framework to directly regress 3-D poses for the candidates, in which a convolutional neural network-based triplet network is trained to extract discriminating features from the binary images. To make the features suitable for the regression task, a pose-guided method and a regression constraint are employed with the constructed triplet network. We show that the constraint reaches the goal of creating the correlation between the features and 3-D poses. Once the expected features are obtained, the object pose could be efficiently regressed, by training a regression network with a simple structure. For symmetric objects, depth images are treated as an additional channel to feed the triplet network. Experiments on the LineMOD and our own datasets demonstrate our method with high regression precision and efficiency.

Mesh denoising is a classical, yet not well-solved problem in digital geometry processing. The challenge arises from noise removal with the minimal disturbance of surface intrinsic properties (e.g., sharp features and shallow details). We propose a new patch normal co-filter (PcFilter) for mesh denoising. It is inspired by the geometry statistics which show that surface patches with similar intrinsic properties exist on the underlying surface of a noisy mesh. We model the PcFilter as a low-rank matrix recovery problem of similar-patch collaboration, aiming at removing different levels of noise, yet preserving various surface features. We generalize our model to pursue the low-rank matrix recovery in the kernel space for handling the nonlinear structure contained in the data. By making use of the block coordinate descent minimization and the specifics of a proximal based coordinate descent method, we optimize the nonlinear and nonconvex objective function efficiently. The detailed quantitative and qualitative results on synthetic and real data show that the PcFilter competes favorably with the state-of-the-art methods in surface accuracy and noise-robustness.

Bas-relief is characterized by its unique presentation of intrinsic shape properties and/or detailed appearance using materials raised up in different degrees above a background. However, many bas-relief modeling methods could not manipulate scene details well. We propose a simple and effective solution for two kinds of bas-relief modeling (i.e., structure-preserving and detail-preserving) which is different from the prior tone mapping alike methods. Our idea originates from an observation on typical 3D models, which are decomposed into a piecewise smooth base layer and a detail layer in normal field. Proper manipulation of the two layers contributes to both structure-preserving and detail-preserving bas-relief modeling. We solve the modeling problem in a discrete geometry processing setup that uses normal-based mesh processing as a theoretical foundation. Specifically, using the two-step mesh smoothing mechanism as a bridge, we transfer the bas-relief modeling problem into a discrete space, and solve it in a least-squares manner. Experiments and comparisons to other methods show that (i) geometry details are better preserved in the scenario with high compression ratios, and (ii) structures are clearly preserved without shape distortion and interference from details.

Abstract The Constructive Solid Geometry (CSG) tree, encoding the generative process of an object by a recursive compositional structure of bounded primitives, constitutes an important structural representation of 3D objects. Therefore, automatically recovering such a compositional structure from the raw point cloud of an object represents a high‐level reverse engineering problem, finding applications from structure and functionality analysis to creative redesign. We propose an effective method to construct CSG models and trees directly over raw point clouds. Specifically, a large number of hypothetical bounded primitive candidates are first extracted from raw scans, followed by a carefully designed pruning strategy. We then choose to approximate the target CSG model by the combination of a subset of these candidates with corresponding Boolean operations using a binary optimization technique, from which the corresponding CSG tree can be derived. Our method attempts to consider the minimal description length concept in the point cloud analysis setting, where the objective function is designed to minimize the construction error and complexity simultaneously. We demonstrate the effectiveness and robustness of our method with extensive experiments on real scan data with various complexities and styles.

While current block diagonal constrained subspace clustering methods are performed explicitly on the original data space, in practice, it is often more desirable to embed the block diagonal prior into the reproducing kernel Hilbert feature space by kernelization techniques, as the underlying data structure in reality is usually nonlinear. However, it is still unknown how to carry out the embedding and kernelization in the models with block diagonal constraints. In this paper, we shall take a step in this direction. First, we establish a novel model termed implicit block diagonal low-rank representation (IBDLR), by incorporating the implicit feature representation and block diagonal prior into the prevalent low-rank representation method. Second, mostly important, we show that the model in IBDLR could be kernelized by making use of a smoothed dual representation and the specifics of a proximal gradient-based optimization algorithm. Finally, we provide some theoretical analyses for the convergence of our optimization algorithm. Comprehensive experiments on synthetic and real-world data sets demonstrate the superiorities of our IBDLR over state-of-the-art methods.

RGB-D video segmentation is important for many applications, including scene understanding, object tracking, and robotic grasping. However, to segment RGB-D frames over a long video sequence into globally consistent segmentation is still a challenging problem. Current methods often lose pixel correspondences between frames under occlusion and, thus, fail to generate consistent and continuous segmentation results. To address this problem, we propose a novel spatiotemporal RGB-D video segmentation framework that automatically segments and tracks objects with continuity and consistency over time. Our approach first produces consistent segments in some keyframes by region clustering, and then propagates the segmentation result to a whole video sequence via a mask propagation scheme in bilateral space. Instead of exploiting local optical, flow information to establish correspondences between adjacent frames, we leverage scale-invariant feature transform (SIFT) flow and bilateral representation to solve inconsistency under occlusion. Moreover, our method automatically extracts multiple objects of interest and tracks them without any user input hint. A variety of experiments demonstrates effectiveness and robustness of our proposed method.

Abstract We present a sparse optimization framework for extracting sparse shape priors from a collection of 3D models. Shape priors are defined as point‐set neighborhoods sampled from shape surfaces which convey important information encompassing normals and local shape characterization. A 3D shape model can be considered to be formed with a set of 3D local shape priors, while most of them are likely to have similar geometry. Our key observation is that the local priors extracted from a family of 3D shapes lie in a very low‐dimensional manifold. Consequently, a compact and informative subset of priors can be learned to efficiently encode all shapes of the same family. A comprehensive library of local shape priors is first built with the given collection of 3D models of the same family. We then formulate a global, sparse optimization problem which enforces selecting representative priors while minimizing the reconstruction error. To solve the optimization problem, we design an efficient solver based on the Augmented Lagrangian Multipliers method (ALM). Extensive experiments exhibit the power of our data‐driven sparse priors in elegantly solving several high‐level shape analysis applications and geometry processing tasks, such as shape retrieval, style analysis and symmetry detection.

The most challenging problem in mesh denoising is to distinguish features from noise. Based on the robust guided normal estimation and alternate vertex updating strategy, we investigate a new feature-preserving mesh denoising method. To accurately capture local structures around features, we propose a corner-aware neighborhood (CAN) scheme. By combining both overall normal distribution of all faces in a CAN and individual normal influence of the interested face, we give a new consistency measuring method, which greatly improves the reliability of the estimated guided normals. As the noise level lowers, we take as guidance the previous filtered normals, which coincides with the emerging rolling guidance idea. In the vertex updating process, we classify vertices according to filtered normals at each iteration and reposition vertices of distinct types alternately with individual regularization constraints. Experiments on a variety of synthetic and real data indicate that our method adapts to various noise, both Gaussian and impulsive, no matter in the normal direction or in a random direction, with few triangles flipped.

Mesh denoising is imperative for improving imperfect surfaces acquired by scanning devices. The main challenge is to faithfully retain geometric features and avoid introducing additional artifacts when removing noise. Unlike the existing mesh denoising techniques that focus only on either the first-order features or high-order differential properties, our approach exploits the synergy when facet normals and quadric surfaces are integrated to recover a piecewise smooth surface. In specific, we vote on surface normal tensors from robust statistics to guide the creation of consistent subneighborhoods subsequently used by moving least squares (MLS). This voting naturally leads to a conceptually simple way that gives a unified mesh-denoising framework for not only handling noise but also enabling the recovering of surfaces with both sharp and small-scale features. The effectiveness of our framework stems from: 1) the multiscale tensor voting that avoids the influence from noise; 2) the effective energy minimization strategy to searching the consistent subneighborhoods; and 3) the piecewise MLS that fully prevents the side effects from different subneighborhoods during surface fitting. Our framework is direct, practical, and easy to understand. Comparisons with the state-of-the-art methods demonstrate its outstanding performance on feature preservation and artifact suppression.

LiDAR scanning has become a prevalent technique for digitalizing large-scale outdoor scenes. However, the raw LiDAR data often contain imperfections, e.g., missing large regions, anisotropy of sampling density, and contamination of noise and outliers, which are the major obstacles that hinder its more ambitious and higher level applications in digital city modeling. Observing that 3D urban scenes can be locally described with several low dimensional subspaces, we propose to locally classify the neighborhoods of the scans to model the substructures of the scenes. The key enabler is the adaptive kernel-scale scoring, filtering and clustering of substructures, making it possible to recover the local structures at all points simultaneously, even in the presence of severe data imperfections. Integrating the local analyses leads to robust shape detection from raw LiDAR data. On this basis, we develop several urban scene applications and verify them on a number of LiDAR scans with various complexities and styles, which demonstrates the effectiveness and robustness of our methods.

We present in this paper an interactive approach for semantically modeling indoor environments given only a single indoor image as input, without requiring access to the scene or using any additional measurements like the RGBD cameras. Our key insight is that, although depth estimation from a single image is notoriously difficult, we can conveniently obtain a relatively accurate normal map, which essentially conveys a great deal of scene geometry. This enables us to model each object in a data-driven manner by representing the object as a normal-based graph and retrieving a similar model from the database by graph matching. We hypothesize a set of sparse surface orientations for the image, and further refine them in an intuitive and straightforward manner. With a small amount of simple user interaction, our approach is able to generate a plausible model of the scene. To verify the effectiveness of our proposed method, we show the modeling results on a variety of indoor images.

Abstract Modeling of urban facades from raw LiDAR point data remains active due to its challenging nature. In this paper, we propose an automatic yet robust 3D modeling approach for urban facades with raw LiDAR point clouds. The key observation is that building facades often exhibit repetitions and regularities. We hereby formulate repetition detection as an energy optimization problem with a global energy function balancing geometric errors, regularity and complexity of facade structures. As a result, repetitive structures are extracted robustly even in the presence of noise and missing data. By registering repetitive structures, missing regions are completed and thus the associated point data of structures are well consolidated. Subsequently, we detect the potential design intents (i.e., geometric constraints) within structures and perform constrained fitting to obtain the precise structure models. Furthermore, we apply structure alignment optimization to enforce position regularities and employ repetitions to infer missing structures. We demonstrate how the quality of raw LiDAR data can be improved by exploiting data redundancy, and discovering high level structural information (regularity and symmetry). We evaluate our modeling method on a variety of raw LiDAR scans to verify its robustness and effectiveness.

Most mesh denoising techniques utilize only either the facet normal field or the vertex normal field of a mesh surface. The two normal fields, though contain some redundant geometry information of the same model, can provide additional information that the other field lacks. Thus, considering only one normal field is likely to overlook some geometric features. In this paper, we take advantage of the piecewise consistent property of the two normal fields and propose an effective framework in which they are filtered and integrated using a novel method to guide the denoising process. Our key observation is that, decomposing the inconsistent field at challenging regions into multiple piecewise consistent fields makes the two fields complementary to each other and produces better results. Our approach consists of three steps: vertex classification, bi-normal filtering, and vertex position update. The classification step allows us to filter the two fields on a piecewise smooth surface rather than a surface that is smooth everywhere. Based on the piecewise consistence of the two normal fields, we filtered them using a piecewise smooth region clustering strategy. To benefit from the bi-normal filtering, we design a quadratic optimization algorithm for vertex position update. Experimental results on synthetic and real data show that our algorithm achieves higher quality results than current approaches on surfaces with multifarious geometric features and irregular surface sampling.

Abstract The emergence of laser/LiDAR sensors, reliable multi‐view stereo techniques and more recently consumer depth cameras have brought point clouds to the forefront as a data format useful for a number of applications. Unfortunately, the point data from those channels often incur imperfection, frequently contaminated with severe outliers and noise. This paper presents a robust consolidation algorithm for low‐quality point data from outdoor scenes, which essentially consists of two steps: 1) outliers filtering and 2) noise smoothing. We first design a connectivity‐based scheme to evaluate outlierness and thereby detect sparse outliers. Meanwhile, a clustering method is used to further remove small dense outliers. Both outlier removal methods are insensitive to the choice of the neighborhood size and the levels of outliers. Subsequently, we propose a novel approach to estimate normals for noisy points based on robust partial rankings, which is the basis of noise smoothing. Accordingly, a fast approach is exploited to smooth noise, while preserving sharp features. We evaluate the effectiveness of the proposed method on the point clouds from a variety of outdoor scenes.

Abstract We propose a robust method for surface mesh reconstruction from unorganized, unoriented, noisy and outlier‐ridden 3D point data. A kernel‐based scale estimator is introduced to estimate the scale of inliers of the input data. The best tangent planes are computed for all points based on mean shift clustering and adaptive scale sample consensus, followed by detecting and removing outliers. Subsequently, we estimate the normals for the remaining points and smooth the noise using a surface fitting and projection strategy. As a result, the outliers and noise are removed and filtered, while the original sharp features are well preserved. We then adopt an existing method to reconstruct surface meshes from the processed point data. To preserve sharp features of the generated meshes that are often blurred during reconstruction, we describe a two‐step approach to effectively recover original sharp features. A number of examples are presented to demonstrate the effectiveness and robustness of our method.

Feature extraction is one of key techniques in feature-based reverse engineering. In this paper, a novel methodology of feature extraction is presented based on collected data points of mechanical part. Firstly, regular surface is used to model individual segmented data points patch based on maximum likelihood estimate. And then the resulting surfaces are used to determine the feature primitives approximately and afterwards extract the feature parameters. Finally, Mahalanobis distance is used to evaluate the error between feature primitives and the resulting surfaces, and feature is positioned optimally utilizing a similarity transformation which minimizes the error.