We present TrackGS, a novel method to integrate global feature tracks with 3D Gaussian Splatting (3DGS) for COLMAP-free novel view synthesis. While 3DGS delivers impressive rendering quality, its reliance on accurate precomputed camera parameters remains a significant limitation. Existing COLMAP-free approaches depend on local constraints that fail in complex scenarios. Our key innovation lies in leveraging feature tracks to establish global geometric constraints, enabling simultaneous optimization of camera parameters and 3D Gaussians. Specifically, we: (1) introduce track-constrained Gaussians that serve as geometric anchors, (2) propose novel 2D and 3D track losses to enforce multi-view consistency, and (3) derive differentiable formulations for camera intrinsics optimization. Extensive experiments on challenging real-world and synthetic datasets demonstrate state-of-the-art performance, with much lower pose error than previous methods while maintaining superior rendering quality. Our approach eliminates the need for COLMAP preprocessing, making 3DGS more accessible for practical applications.

We present a novel framework for automated interior design that combines large language models (LLMs) with grid-based integer programming to jointly optimize room layout and furniture placement. Given a textual prompt, the LLM-driven agent workflow extracts structured design constraints related to room configurations and furniture arrangements. These constraints are encoded into a unified grid-based representation inspired by ``Modulor". Our formulation accounts for key design requirements, including corridor connectivity, room accessibility, spatial exclusivity, and user-specified preferences. To improve computational efficiency, we adopt a coarse-to-fine optimization strategy that begins with a low-resolution grid to solve a simplified problem and guides the solution at the full resolution. Experimental results across diverse scenarios demonstrate that our joint optimization approach significantly outperforms existing two-stage design pipelines in solution quality, and achieves notable computational efficiency through the coarse-to-fine strategy.

Abstract Creating floorplans lays the foundation for architectural design and scene modeling. We propose a novel framework for generating diverse high‐quality floorplans under predefined constraints. Central to our method is an iterative refinement process for optimizing the bounding boxes of rooms and the floorplan semantics image, which defines a vector floorplan together. Vector floorplans can be generated through a learning‐based refinement process. Our framework supports various constraints, such as floorplan boundaries, topological graphs, and bubble diagrams. Extensive experiments demonstrate that our method is superior to state‐of‐the‐art techniques, particularly in generating a wider variety of solutions that cater to various architectural needs.

Abstract Tessellation is critical for real‐time rendering of Catmull‐Clark subdivision (CCS) surfaces, with control mesh singularities posing a major challenge. Prior works typically pre‐subdivide singular regions to a fixed depth offline to facilitate efficient runtime tessellation, yet this practice fails to meet view‐dependent error requirements for high‐fidelity rendering—especially at pixel‐level accuracy. Hence, we present EBOAT, a novel direct‐evaluation tessellation method that integrates memory‐efficient feature‐adaptive subdivision via a connectivity‐elided representation for singularities, along with a two‐phase error control strategy and improved subdivision depth estimation built upon it. With a uniform tessellation level of 64, EBOAT achieves a 3.8×–4.7× speedup over OpenSubdiv, without relying on fixed‐depth pre‐subdivision. Compared to state‐of‐the‐art breadth‐first subdivision, it yields a 4.2×–7.1× speedup while providing more accurate sampling. To further demonstrate its effectiveness, we implement a pixel‐accurate CCS surface rendering pipeline with EBOAT fully integrated, and provide an in‐depth evaluation of rendering quality. Given its advantages in both performance and accuracy, EBOAT offers a promising alternative for real‐time CCS surface rendering in industrial applications.

Abstract We present ParaMG, a surface multigrid method that restores classical multigrid structure on curved surfaces by expressing all hierarchy levels in a single globally consistent planar domain. Existing surface multigrid methods rely on composed local parameterizations or 3D projections for cross‐level transfer, requiring repeated per‐level local approximations that can degrade convergence. Our key insight is that a global conformal parameterization with cone singularities provides a shared coordinate system with low, controllable distortion, where prolongation reduces to exact planar barycentric interpolation. We construct the hierarchy directly in this domain using a seam‐aware simplification strategy, yielding sparse prolongation matrices with three entries per row. ParaMG converges in fewer V‐cycles than prior surface multigrid methods and delivers substantial speedups in applications with changing linear systems, including geometric flows, thin‐shell simulation, and polycube deformation.

Integrating artificial intelligence (AI) into computer-aided design (CAD) has shown the potential to transform design and manufacturing processes, enabling more efficient, intuitive, and intelligent workflows. In recent years, the application of AI to 3D CAD model generation tasks has gradually emerged. To better enable researchers to understand the current research status of the AI-based CAD generation field and to inspire them to conduct further research, this survey explores the role of AI in 3D CAD model generation tasks that utilize various representations and conditions, ranging from traditional machine learning to LLM-based approaches. Additionally, AI applications in other extended CAD areas are also touched upon in the survey. Finally, we analyze current progress, identify challenges and limitations faced by this field, and propose possible directions for future work.

3D shape segmentation is a fundamental problem in computer vision, supporting applications such as shape reconstruction and semantic understanding. A persistent challenge in learning-based methods is the degradation of performance near part boundaries, commonly attributed to class-level imbalance. In this work, we reveal that the primary source of this issue instead arises from a pronounced imbalance between marginal and internal areas within partitions in 3D shapes, which is largely overlooked by existing methods and leads to systematically poor boundary discrimination. To address this problem, we propose a marginal-aware segmentation framework that explicitly emphasizes boundary localization and relational modeling. The framework is realized by two implementations that serve complementary purposes. Specifically, the staged variant emphasizes interpretability by identifying marginal faces and refining them through topology-aware subgraphs and a Graph Attention Network (GAT), while the end-to-end differentiable integration incorporates our marginal-aware modeling principle into a modern 3D segmentation pipeline (SAMPart3D) by leveraging SAM-derived boundary cues to demonstrate performance gains on state-of-the-art systems. Extensive experiments on PSB, COSEG, and HumanBody demonstrate that the staged variant substantially improves boundary recognition, yielding a 12.32% gain in boundary accuracy over state-of-the-art methods, while the end-to-end integration further improves mIoU from 53.7% to 55.3% on PartObjaverse-Tiny with a modest computational overhead. These results demonstrate that explicit marginal-aware relational modeling is an effective and flexible strategy for improving 3D shape segmentation, particularly in challenging boundary regions.

Similar to facial beautification in real life, 3D virtual avatars require personalized customization to enhance their visual appeal, yet this area remains insufficiently explored. Although current 3D Gaussian editing methods can be adapted for facial makeup purposes, these methods fail to meet the fundamental requirements for achieving realistic makeup effects: 1) ensuring a consistent appearance during drivable expressions; 2) preserving the identity throughout the makeup process; and 3) enabling precise control over fine details. To address these, we propose a specialized 3D makeup method named AvatarMakeup, leveraging a pretrained diffusion model to transfer makeup patterns from a single reference photo of any individual. We adopt a coarse-to-fine idea to first maintain the consistent appearance and identity, and then to refine the details. In particular, the diffusion model is employed to generate makeup images as supervision. Due to the uncertainties in diffusion process, the generated images are inconsistent across different viewpoints and expressions. Therefore, we propose a Coherent Duplication method to coarsely apply makeup to the target while ensuring consistency across dynamic and multi-view effects. Coherent Duplication optimizes a global UV map by recoding the averaged facial attributes among the generated makeup images. By querying the global UV map, it easily synthesizes coherent makeup guidance from arbitrary views and expressions to optimize the target avatar. Given the coarse makeup avatar, we further enhance the makeup by incorporating a Refinement Module into the diffusion model to achieve high makeup quality. Experiments demonstrate that AvatarMakeup achieves state-of-the-art makeup transfer quality and consistency throughout animation.

Point cloud reconstruction is an ingredient in geometry modeling, computer graphics, and 3D vision. In this paper, we propose a novel unsupervised learning method called the Recurrent Multi-Step Moving Strategy, which progressively moves query points toward the underlying surface to accurately learn unsigned distance fields (UDFs) for point cloud reconstruction. Specifically, we design a recurrent network for UDF estimation that integrates a multi-step strategy for query movement. This model treats query movement as a trajectory prediction process, establishing dependencies between the current query move decision and the previous path, thus utilizing temporal information to improve UDF estimation accuracy. Further, we design distance and gradient regularization losses to ensure the precision, consistency, and continuity of the estimated UDFs. Extensive evaluations, comparisons, and ablation studies are conducted to show the superiority of our method over the competing approaches in terms of reconstruction accuracy and generality. Our unsupervised reconstruction method outperforms many supervised techniques and demonstrates efficacy across diverse scenarios, including single-object, indoor, and outdoor benchmarks.

The Walk on Spheres (WoS) algorithm is a mesh-free and highly flexible Monte Carlo method for solving partial differential equations, but its practical applicability is limited by slow O ( N -1/2 ) convergence. While prior variance reduction techniques exploit spatial correlations through integral properties of the PDE, they do not fully utilize the intrinsic Markov structure of the WoS process. We introduce a new variance reduction framework based on reusing intermediate states along each random walk. Leveraging the Markov property, we show that every point visited by a WoS trajectory provides a valid unbiased estimator, but its direct use is hindered by the complex distribution induced by dynamically generated spheres. To resolve this, we propose the Walk on Probes (WoP) algorithm, which replaces dynamic spheres with a set of fixed, pre-distributed spherical probes inside the domain. This converts the intractable distribution of path points into samples on fixed boundaries, enabling efficient evaluation through the Poisson integral formula. We further develop a specialized method that combines control variates with self-normalization to further reduce variance. Together, these components substantially improve sample efficiency while preserving the flexibility of WoS. Code and data for this paper are at https://github.com/USTCGCL-WoS/Walk-on-Probes.

Reconstructing transparent objects with high fidelity presents significant challenges due to complex light refraction and reflection. Existing methods rely on intentionally designed patterns observed behind the transparent object to infer the correspondence between rays and the background, thereby improving the precision of the reconstruction. However, they are hindered by a refraction-tracing-based strategy that fails to reconstruct complex nested transparent objects and a tedious view-capture strategy relying on images captured from empirically determined viewpoints. To overcome these obstacles, we propose AHC-NeRF, an autonomous, high-quality neural SDF-based framework designed for reconstructing two-layer complex nested transparent objects. Firstly, our framework combines neural SDF with single-pixel imaging, a reflection-based method, which utilizes point-pair priors as guidance to achieve high-quality reconstruction of both the outer and inner surfaces. Secondly, we propose an adaptive single-pixel imaging method that achieves an acceleration of 1-2 orders of magnitude compared to vanilla single-pixel imaging for the acquisition of point-pair priors. Finally, we introduce a novel view-planning strategy that progressively identifies the viewpoints with the highest information gain throughout the optimization process, thereby achieving high-quality surface reconstruction. Extensive experimental results on both synthetic and real-world datasets demonstrate that AHC-NeRF outperforms state-of-the-art methods.

We propose an efficient method to compute a small set of integer-constrained cone singularities, which induce a rotationally seamless conformal parameterization with low distortion. Since the problem only involves discrete variables, i.e., vertex-constrained positions, integer-constrained angles, and the number of cones, we alternately optimize these three types of variables to achieve tractable convergence. Central to high efficiency is an explicit construction algorithm that reduces the optimization problem scale to be slightly greater than the number of integer variables for determining the optimal angles with fixed positions and numbers, even for high-genus surfaces. In addition, we derive a new derivative formula that allows us to move the cones, effectively reducing distortion until convergence. Combined with other strategies, including repositioning and adding cones to decrease distortion, adaptively selecting a constrained number of integer variables for efficient optimization, and pairing cones to reduce the number, we quickly achieve a favorable tradeoff between the number of cones and the parameterization distortion. We demonstrate the effectiveness and practicability of our cones by using them to generate rotationally seamless and low-distortion parameterizations on a massive test data set. Our method demonstrates an order-of-magnitude speedup (30× faster on average) compared to state-of-the-art approaches while maintaining comparable cone numbers and parameterization distortion.

In modern computer-aided design (CAD), Boundary Representation (B-rep) is a widely used geometric modeling technique in industrial design and manufacturing. However, existing B-rep generation methods, which rely on tree-based hierarchies to represent and generate B-reps, fail to fully exploit the inherent graph structure of B-reps, resulting in suboptimal efficiency and model quality. To address this issue, we propose BRep-GD, a graph diffusion-based model specifically designed for B-rep generation. Unlike prior methods, BRep-GD treats B-reps as graphs, where nodes represent face elements, and edges represent boundary and vertex elements. By utilizing a continuous topological graph data structure, BRep-GD overcomes the challenges associated with directly applying graph diffusion models to B-rep generation. Specifically, BRep-GD introduces a graph diffusion method tailored to the features of CAD data, generating faces and edges sequentially. During edge generation, continuous topology decoupling is employed to avoid the need for global attention calculations, reducing computational complexity while ensuring geometric consistency and high-quality results. Experimental results demonstrate that BRep-GD outperforms existing state-of-the-art methods in both unconditional and class-conditional generation tasks, particularly in generating watertight solids and handling complex geometries. It significantly reduces isolated or inconsistent geometric components and improves generation efficiency.

Biharmonic coordinates have become a powerful tool for cage-based deformation, owing to their inherent interpolation properties. However, their derivation for polynomial cages in 3D has remained unsolved. To address this, we propose closed-form expressions for polynomial 3D biharmonic coordinates and their derivatives when deformed from polygonal cages using the high-order boundary element method. Our primary contribution lies in the analytical derivation of the kernel integration using recursive differentiation techniques. Due to the enriched deformation space of biharmonic coordinates and the flexibility of polynomial cages, our method supports a broad range of deformations, as demonstrated through extensive experiments.

We propose a novel method to generate a small set of ruled surfaces that do not collide with the input shape for linear hot-wire rough machining. Central to our technique is a new observation: ruled surfaces constructed by vertical extrusion from planar smooth curves that approach the input shape's outer contour lines without collisions can effectively remove material during rough machining. Accordingly, we develop an iterative algorithm that alternates in each iteration between computing a viewpoint to determine an outer contour line and optimizing a smooth curve to approximate that contour line under the collision-free constraint. Specifically, a view selection approach based on a genetic algorithm is used to optimize the viewpoint for removing materials as much as possible, and an adaptive fitting algorithm is presented to find the constrained curves. The feasibility and practicability of our method are demonstrated through 10 physical examples. Compared with manual designs, our method obtains lower errors with the same number of cuts.

Imperceptible adversarial attacks on 3D point clouds rely on effective constraints. While manifold constraints have notable advantages over Euclidean ones, the global parameterization used in current methods often fails to fully preserve manifold properties. In this paper, we propose to constrain lattice-based barycentric coordinates during attacks from a local parametric perspective to ensure imperceptibility. Specifically, we utilize a permutohedral lattice to partition point clouds into multiple cells, and then extract barycentric coordinates for each point within these cells, forming a local parametric representation of the point clouds. By enforcing local parametric constraints that minimize the displacement of barycentric coordinates, we largely preserve the manifold properties, ultimately leading to improved imperceptibility. Extensive experiments validate that integrating these local parametric constraints into conventional adversarial attacks yields superior imperceptibility, outperforming state-of-the-art methods.

We tackle the challenge of concurrent reconstruction at the part level with the RGB appearance and estimation of motion parameters for building digital twins of articulated objects using the 3D Gaussian Splatting (3D-GS) method. With two distinct sets of multi-view imagery, each depicting an object in separate static articulation configurations, we reconstruct the articulated object in 3D Gaussian representations with both appearance and geometry information at the same time. Our approach decoupled multiple highly interdependent parameters through a multi-step optimization process, thereby achieving a stable optimization procedure and high-quality outcomes. We introduce ArticulatedGS, a self-supervised, comprehensive framework that autonomously learns to model shapes and appearances at the part level and synchronizes the optimization of motion parameters, all without reliance on 3D supervision, motion cues, or semantic labels. Our experimental results demonstrate that, among comparable methodologies, our approach has achieved optimal outcomes in terms of part segmentation accuracy, motion estimation accuracy, and visual quality. The code will be made publicly available at our website https://guojunfu-tech.github.io/articulatedGS-io/

Generating high-quality novel view renderings of 3D Gaussian Splatting (3DGS) in scenes featuring transient objects is challenging. We propose a novel hybrid representation, termed as HybridGS, using 2D Gaussians for transient objects per image and maintaining traditional 3D Gaussians for the whole static scenes. 3DGS is suited for modeling static scenes that assume multi-view consistency, but the transient objects appear occasionally and do not adhere to the assumption, thus we model them as planar objects from a single view by 2D Gaussians. Our novel representation decomposes the scene from the perspective of fundamental viewpoint consistency. Additionally, we present a multi-view supervision method for 3DGS that leverages information from co-visible regions, further enhancing the distinctions between the transients and statics. Then, we propose a straightforward yet effective multi-stage training strategy to ensure robust training and view synthesis. Experiments on benchmarks show our state-of-the-art performance of novel view synthesis in indoor and outdoor scenes, even in the presence of distracting elements. Project page: https://gujiaqivadin.github.io/hybridgs/

Computer-Aided Design (CAD) generative modeling is driving significant innovations across industrial applications. Recent works have shown remarkable progress in creating solid models from various inputs such as point clouds, meshes, and text descriptions. However, these methods fundamentally diverge from traditional industrial workflows that begin with 2D engineering drawings. The automatic generation of parametric CAD models from these 2D vector drawings remains underexplored despite being a critical step in engineering design. To address this gap, our key insight is to reframe CAD generation as a sequence-to-sequence learning problem where vector drawing primitives directly inform the generation of parametric CAD operations, preserving geometric precision and design intent throughout the transformation process. We propose Drawing2CAD, a framework with three key technical components: a network-friendly vector primitive representation that preserves precise geometric information, a dual-decoder transformer architecture that decouples command type and parameter generation while maintaining precise correspondence, and a soft target distribution loss function accommodating inherent flexibility in CAD parameters. To train and evaluate Drawing2CAD, we create CAD-VGDrawing, a dataset of paired engineering drawings and parametric CAD models, and conduct thorough experiments to demonstrate the effectiveness of our method. Code and dataset are available at https://github.com/lllssc/Drawing2CAD.

We propose closed-form Cauchy coordinates and their derivatives for 2D closed high-order input cages composed of arbitrary-order polynomial curves. Our coordinates facilitate the transformation of input polynomial curves into output curves of any desired polynomial order. Central to our derivation is the creative use of the residue theorem with the logarithmic function to obtain the integral of a rational polynomial required for extending the classical 2D Cauchy coordinates to high-order input cages. Our coordinates enable smooth cage-aware angle-preserving deformations, and the derivatives allow for point-to-point deformation. Moreover, our derivation can be extended to the input cages with rational polynomial curves. Through various 2D deformations, we demonstrate how users can intuitively manipulate Bézier control points to achieve desired deformations easily.

Abstract We propose a novel method to filter eigenvalues for projected Newton. Central to our method is blending the clamped and absolute eigenvalues to adaptively compute the modified Hessian matrix. To determine the blending coefficients, we rely on (1) a key observation and (2) an objective function descent constraint. The observation is that if the quadratic form defined by the Hessian matrix maps the descent direction to a negative real number, the decrease in the objective function is limited. The constraint is that our eigenvalue filtering leads to more reduction in objective function than the absolute eigenvalue filtering [CLL*24] in the case of second‐order Taylor approximation. Our eigenvalue blending is easy to implement and leads to fewer optimization iterations than the state‐of‐the‐art eigenvalue filtering methods.

Abstract In this paper, we introduce NoPe‐NeRF++, a novel local‐to‐global optimization algorithm for training Neural Radiance Fields (NeRF) without requiring pose priors. Existing methods, particularly NoPe‐NeRF, which focus solely on the local relationships within images, often struggle to recover accurate camera poses in complex scenarios. To overcome the challenges, our approach begins with a relative pose initialization with explicit feature matching, followed by a local joint optimization to enhance the pose estimation for training a more robust NeRF representation. This method significantly improves the quality of initial poses. Additionally, we introduce global optimization phase that incorporates geometric consistency constraints through bundle adjustment, which integrates feature trajectories to further refine poses and collectively boost the quality of NeRF. Notably, our method is the first work that seamlessly combines the local and global cues with NeRF, and outperforms state‐of‐the‐art methods in both pose estimation accuracy and novel view synthesis. Extensive evaluations on benchmark datasets demonstrate our superior performance and robustness, even in challenging scenes, thus validating our design choices.

Although supervised deep normal estimators have recently shown impressive results on synthetic benchmarks, their performance deteriorates significantly in real-world scenarios due to the domain gap between synthetic and real data. Building high-quality real training data to boost those supervised methods is not trivial because point-wise annotation of normals for varying-scale real-world 3D scenes is a tedious and expensive task. This paper introduces PointNorm-Net, the first self-supervised deep learning framework to tackle this challenge. The key novelty of PointNorm-Net is a three-stage multi-modal normal distribution estimation paradigm that can be integrated into either deep or traditional optimization-based normal estimation frameworks. Extensive experiments show that our method achieves superior generalization and outperforms state-of-the-art conventional and deep learning approaches across three real-world datasets that exhibit distinct characteristics compared to the synthetic training data.

3D shape segmentation is a crucial task in the field of multimedia analysis and processing, and recent years have seen a surge in research on this topic. However, many existing methods only consider geometric features of 3D shapes and fail to explore the potential connections between faces, limiting their segmentation performance. In this paper, we propose a novel segmentation approach that mines and enhances the potential consistency of 3D shapes to overcome this limitation. The key idea is to mine the consistency between different partitions of 3D shapes and to use the unique consistency enhancement strategy to continuously optimize the consistency features for the network. Our method also includes a comprehensive set of network structures to mine and enhance consistent features, enabling more effective feature extraction and better utilization of contextual information around each face when processing complex shapes. We evaluate our approach on public benchmarks through extensive experiments and demonstrate its effectiveness in achieving higher accuracy than existing methods.

We derive closed-form expressions of biharmonic coordinates for 2D high-order cages, enabling the transformation of the input polynomial curves into polynomial curves of any order. Central to our derivation is the use of the high-order boundary element method. We demonstrate the practicality and effectiveness of our method on various 2D deformations. In practice, users can easily manipulate the Bézier control points to perform the desired intuitive deformation, as the biharmonic coordinates provide an enriched deformation space and encourage the alignment between the boundary cage and its interior geometry.

We derive closed-form expressions for generalized winding numbers of rational parametric curves for robust containment queries. Given an oriented rational parametric curve and a query point, the generalized winding number can be reformulated to an integral of a rational polynomial. The key to computing the integral lies in using the residue theorem. Then, add up the contributions of each curve to obtain the generalized winding numbers of a set of rational parametric curves. Furthermore, the derivatives of generalized winding numbers are easily derived. Consequently, the expressions for generalized winding numbers are concise and computationally efficient, becoming faster than state-of-the-art methods. Moreover, the computational costs for various query points are almost the same.

Watertight tessellation is essential for real-time rendering of large-scale surfaces, particularly for Non-Uniform Rational B-Splines (NURBS) and Catmull-Clark Subdivision (CCS) surfaces. We present WATER, a software-based framework that delivers watertight, non-uniform tessellation with pixel-level accuracy at real-time frame rates. Unlike fixed-function hardware tessellation, WATER adopts a fully GPU-driven pipeline with cache-friendly design and novel algorithms, offering greater flexibility, scalability, and performance. Under our framework, a 2×–3× speedup over hardware tessellation is achieved for bi-3 Bézier surfaces, while bi-7 Bézier surfaces exhibit a 7×–11× improvement. Compared to ETER [Xiong et al. 2023], our method achieves 1.3×–2.1× faster rendering and 52%–72% lower memory usage under the same non-watertight uniform pattern. When enforcing watertightness, a moderate overhead of 23%–51% is incurred. With its advantages in quality, efficiency, and adaptability, WATER provides a compelling alternative for industrial-scale rendering tasks requiring watertightness.

We present an asymptotic analysis of shell lattice metamaterials based on Ciarlet's shell theory, introducing a new metric— asymptotic directional stiffness (ADS)—to quantify how the geometry of the middle surface governs the effective stiffness. We prove a convergence theorem that rigorously characterizes ADS and establishes its upper bound, along with necessary and sufficient condition for achieving it. As a key result, our theory provides the first rigorous explanation for the high bulk modulus observed in Triply Periodic Minimal Surfaces (TPMS)-based shell lattices. To optimize ADS on general periodic surfaces, we propose a triangular-mesh-based discretization and shape optimization framework. Numerical experiments validate the theoretical findings and demonstrate the effectiveness of the optimization under various design objectives. Our implementation is available at https://github.com/lavenklau/minisurf.

Generating artistic and coherent 3D scene layouts is crucial in digital content creation. Traditional optimization-based methods are often constrained by cumbersome manual rules, while deep generative models face challenges in producing content with richness and diversity. Furthermore, approaches that utilize large language models frequently lack robustness and fail to accurately capture complex spatial relationships. To address these challenges, this paper presents a novel vision-guided 3D layout generation system. We first construct a high-quality asset library containing 2,037 scene assets and 147 3D scene layouts. Subsequently, we employ an image generation model to expand prompt representations into images, fine-tuning it to align with our asset library. We then develop a robust image parsing module to recover the 3D layout of scenes based on visual semantics and geometric information. Finally, we optimize the scene layout using scene graphs and overall visual semantics to ensure logical coherence and alignment with the images. Extensive user testing demonstrates that our algorithm significantly outperforms existing methods in terms of layout richness and quality. The code and dataset will be available at https://github.com/HiHiAllen/Imaginarium.

We propose a novel method to generate developable approximations for triangular meshes. Instead of fitting the Gauss image using a geodesic circle in the local neighborhood, we apply a nonlinear dimensionality reduction method, called Isomap, to use a general curve on the sphere for fitting. This brings us a larger space to represent the Gauss image in the local neighborhood as a 1D structure. Specifically, each triangle is assigned a target normal after local fitting; then, we deform the mesh to approach the target normal globally. By iteratively performing fitting and deformation, we obtain the developable approximation. We demonstrate the feasibility and effectiveness of our method over various examples. Compared to the state-of-the-art methods, our results exhibit a higher fidelity to the input mesh while possessing more prominent and visually distinct undevelopable seam curves.

We propose a novel method to robustly and efficiently compute the maximum allowable step sizes so that the 3D high-order finite elements continuously preserve geometric validity when moving along the given directions with positive step sizes smaller than the computed ones. We transform the problem of finding the maximum allowable step sizes to one of solving roots of cubic polynomials. To use interval arithmetic to avoid numerical issues in cubic equation solving, we completely enumerate the roots of cubic polynomials and apply the interval version of the Newton-Raphson iteration. The effectiveness of our algorithm is demonstrated through extensive testing. Compared to the state-of-the-art method, our algorithm achieves higher efficiency.

The task of 3D shape captioning occupies a significant place within the domain of computer graphics and has garnered considerable interest in recent years. Traditional approaches to this challenge frequently depend on the utilization of costly voxel representations or object detection techniques, yet often fail to deliver satisfactory outcomes. To address the above challenges, in this paper, we introduce Diff-3DCap, which employs a sequence of projected views to represent a 3D object and a continuous diffusion model to facilitate the captioning process. More precisely, our approach utilizes the continuous diffusion model to perturb the embedded captions during the forward phase by introducing Gaussian noise and then predicts the reconstructed annotation during the reverse phase. Embedded within the diffusion framework is a commitment to leveraging a visual embedding obtained from a pre-trained visual-language model, which naturally allows the embedding to serve as a guiding signal, eliminating the need for an additional classifier. Extensive results of our experiments indicate that Diff-3DCap can achieve performance comparable to that of the current state-of-the-art methods.

Identifying points of interest (POIs) on the surface of 3D shapes is a significant challenge in geometric processing research. The complex connection between POIs and their geometric descriptors, combined with the small percentage of POIs on the shape, makes detecting POIs on any given 3D shape a highly challenging task. Existing methods directly detect POIs from the entire 3D shape, resulting in low efficiency and accuracy. Therefore, we propose a novel multi-modal POI detection method using a coarse-to-fine approach, with the key idea of reducing data complexity and enabling more efficient and accurate subsequent POI detection by first identifying and processing important regions on the 3D shape. It first obtains important areas on the 3D shape through 2D projected images, then processes points within these regions using attention mechanisms. Extensive experiments demonstrate that our method outperforms existing POI detection techniques.

This paper presents GaussEdit, a framework for adaptive 3D scene editing guided by text and image prompts. GaussEdit leverages 3D Gaussian Splatting as its backbone for scene representation, enabling convenient Region of Interest selection and efficient editing through a three-stage process. The first stage involves initializing the 3D Gaussians to ensure high-quality edits. The second stage employs an Adaptive Global-Local Optimization strategy to balance global scene coherence and detailed local edits and a category-guided regularization technique to alleviate the Janus problem. The final stage enhances the texture of the edited objects using a sophisticated image-to-image synthesis technique, ensuring that the results are visually realistic and align closely with the given prompts. Our experimental results demonstrate that GaussEdit surpasses existing methods in editing accuracy, visual fidelity, and processing speed. By successfully embedding user-specified concepts into 3D scenes, GaussEdit is a powerful tool for detailed and user-driven 3D scene editing, offering significant improvements over traditional methods.

Abstract We propose a novel method for generating symmetric piecewise developable approximations for shapes in approximately global reflectional or rotational symmetry. Given a shape and its symmetry constraint, the algorithm contains two crucial steps: (i) a symmetric deformation to achieve a nearly developable model and (ii) a symmetric segmentation aided by the deformed shape. The key to the deformation step is the use of the symmetric implicit neural representations of the shape and the deformation field. A new mesh extraction from the implicit function is introduced to construct a strictly symmetric mesh for the subsequent segmentation. The symmetry constraint is carefully integrated into the partition to achieve the symmetric piecewise developable approximation. We demonstrate the effectiveness of our algorithm over various meshes.

Abstract We introduce a novel framework for surface cutting and flattening, aiming to align the boundary of planar parameterization with a target shape. Diverging from traditional methods focused on minimizing distortion, we intend to also achieve shape similarity between the parameterized mesh and a specific planar target, which is important in some applications of art design and texture mapping. However, with existing methods commonly limited to ellipsoidal surfaces, it still remains a challenge to solve this problem on general surfaces. Our framework models the general case as a joint optimization of cuts and parameterization, guided by a novel metric assessing shape similarity. To circumvent the common issue of local minima, we introduce an extra global seam updating strategy which is guided by the target shape. Experimental results show that our framework not only aligns with previous approaches on ellipsoidal surfaces but also achieves satisfactory results on more complex ones.

Abstract In real‐time rendering, optimizing the sampling of large‐scale candidates is crucial. The spatiotemporal reservoir resampling (ReSTIR) method provides an effective approach for handling large candidate samples, while the Generalized Resampled Importance Sampling (GRIS) theory provides a general framework for resampling algorithms. However, we have observed that when using the generalized multiple importance sampling (MIS) weight in previous work during spatiotemporal reuse, variances gradually amplify in the candidate domain when there are significant differences. To address this issue, we propose a new MIS weight suitable for resampling that blends samples from different sampling domains, ensuring convergence of results as the proportion of non‐canonical samples increases. Additionally, we apply this weight to temporal resampling to reduce noise caused by scene changes or jitter. Our method effectively reduces energy loss in the biased version of ReSTIR DI while incurring no additional overhead, and it also suppresses artifacts caused by a high proportion of temporal samples. As a result, our approach leads to lower variance in the sampling results.

3D shape segmentation is a fundamental and crucial task in the field of image processing and 3D shape analysis. To segment 3D shapes using data-driven methods, a fully labeled dataset is usually required. However, obtaining such a dataset can be a daunting task, as manual face-level labeling is both time-consuming and labor-intensive. In this paper, we present a semi-supervised framework for 3D shape segmentation that uses a small, fully labeled set of 3D shapes, as well as a weakly labeled set of 3D shapes with sparse scribble labels. Our framework first employs an auxiliary network to generate initial fully labeled segmentation labels for the sparsely labeled dataset, which helps in training the primary network. During training, the self-refine module uses increasingly accurate predictions of the primary network to improve the labels generated by the auxiliary network. Our proposed method achieves better segmentation performance than previous semi-supervised methods, as demonstrated by extensive benchmark tests, while also performing comparably to supervised methods.

Image translation has various applications in computer graphics and computer vision, aiming to transfer images from one domain to another. Thanks to the excellent generation capability of diffusion models, recent single-view image translation methods achieve realistic results. However, directly applying diffusion models for multi-view image translation remains challenging for two major obstacles: the need for paired training data and the limited view consistency. To overcome the obstacles, we present a first unified multi-view image to multi-view image translation framework based on diffusion models, called MV2MV. Firstly, we propose a novel self-supervised training strategy that exploits the success of off-the-shelf single-view image translators and the 3D Gaussian Splatting (3DGS) technique to generate pseudo ground truths as supervisory signals, leading to enhanced consistency and fine details. Additionally, we propose a latent multi-view consistency block, which utilizes the latent-3DGS as the underlying 3D representation to facilitate information exchange across multi-view images and inject 3D prior into the diffusion model to enforce consistency. Finally, our approach simultaneously optimizes the diffusion model and 3DGS to achieve a better trade-off between consistency and realism. Extensive experiments across various translation tasks demonstrate that MV2MV outperforms task-specific specialists in both quantitative and qualitative.

A mechanism is an assembly of moving parts interconnected by joints to transfer an input motion to a desired output motion. Traditionally, to generate a complex motion, mechanisms are modeled by selecting and combining a number of mechanical parts with simple shapes such as links, gears, and cams. Combining multiple mechanical parts results in a mechanism with an intricate topology, which not only complicates assembly and maintenance but also deteriorates the functionality of generating motions due to accumulation of manufacturing imprecisions. To get rid of these limitations, we study mechanisms with a single pair of moving parts for generating complex motions. We model the pair of moving parts as a pair of conjugate surfaces with multiple conjugation points, forming a multi-point conjugation mechanism. To study this new mechanism, we establish a connection between conjugate surface pairs and form-closure grasps to formulate a dynamic form closure condition under which one conjugate surface is able to continuously transfer the motion to the other conjugate surface by utilizing multiple conjugation points. Guided by the condition, we propose an optimization-based approach to model the geometry of a multi-point conjugation mechanism for exactly generating a user-specified motion, in 1-, 2-, or 3-DOF motion space. The core of our approach is to model multiple conjugate curve pairs that satisfy various requirements in multi-point conjugation, dynamic form closure, and surface fabricability. We demonstrate the effectiveness of our approach by modeling different classes of multi-point conjugation mechanisms to generate various motions, evaluating the mechanisms' kinematic performance with 3D printed prototypes, and presenting three applications of these mechanisms.

We propose a simple yet effective method to orient normals for point clouds. Central to our approach is a novel optimization objective function defined from global and local perspectives. Globally, we introduce a signed uncertainty function that distinguishes the inside and outside of the underlying surface. Moreover, benefiting from the statistics of our global term, we present a local orientation term instead of a global one. The optimization problem can be solved by the commonly used numerical optimization solver, such as L-BFGS. The capability and feasibility of our approach are demonstrated over various complex point clouds. We achieve higher practical robustness and normal quality than the state-of-the-art methods.

We propose a new structure called a higher-order shell, which is composed of a set of triangular prisms. Each triangular prism is enveloped by three Bézier triangles (top, middle, and bottom) and three side surfaces, each of which is trimmed from a bilinear surface. Moreover, we define a continuous vector field to smoothly and bijectively transfer attributes between two surfaces inside the shell. Since the higher-order shell has several hard construction constraints, we apply an interior-point strategy to robustly and automatically construct a high-order shell for an input mesh. Specifically, the strategy starts from a valid linear shell with a small thickness. Then, the shell is optimized until the specified thickness is reached, where explicit checks ensure that the constraints are always satisfied. We extensively test our method on more than 8300 models, demonstrating its robustness and performance. Compared to state-of-the-art methods, our bijective projection is smoother, and the space between the shell and input mesh is more uniform.

Designing thin-shell structures that are diverse, lightweight, and physically viable is a challenging task for traditional heuristic methods. To address this challenge, we present a novel parametric design framework for engraving regular, irregular, and customized patterns on thin-shell structures. Our method optimizes pattern parameters such as size and orientation, to ensure structural stiffness while minimizing material consumption. Our method is unique in that it works directly with shapes and patterns represented by functions, and can engrave patterns through simple function operations. By eliminating the need for remeshing in traditional FEM methods, our method is more computationally efficient in optimizing mechanical properties and can significantly increase the diversity of shell structure design. Quantitative evaluation confirms the convergence of the proposed method. We conduct experiments on regular, irregular, and customized patterns and present 3D printed results to demonstrate the effectiveness of our approach.

We propose a novel and automatic method to model shapes using a small set of discrete developable patches. Central to our approach is using implicit neural shape representation that makes our algorithm independent of tessellation and allows us to obtain the Gaussian curvature of each point analytically. With this powerful representation, we first deform the input shape to be an almost developable shape with clear and sparse salient feature curves. Then, we convert the deformed implicit field to a triangle mesh, which is further cut to disk topology along parts of the sparse feature curves. Finally, we achieve the resulting piecewise developable mesh by alternatingly optimizing discrete developability, enforcing manufacturability constraints, and merging patches. The feasibility and practicability of our method are demonstrated over various shapes. Compared to the state-of-the-art methods, our method achieves a better tradeoff between the number of developable patches and the approximation error.

Structural design with multi-family triply periodic minimal surfaces (TPMS) is a meaningful work that can combine the advantages of different types of TPMS. However, very few methods consider the influence of the blending of different TPMS on structural performance, and the manufacturability of final structure. Therefore, this work proposes a method to design manufacturable microstructures with topology optimization (TO) based on spatially-varying TPMS. In our method, different types of TPMS are simultaneously considered in the optimization to maximize the performance of designed microstructure. The geometric and mechanical properties of the unit cells generated with TPMS, that is minimal surface lattice cell (MSLC), are analyzed to obtain the performance of different types of TPMS. In the designed microstructure, MSLCs of different types are smoothly blended with an interpolation method. To analyze the influence of deformed MSLCs on the performance of the final structure, the blending blocks are introduced to describe the connection cases between different types of MSLCs. The mechanical properties of deformed MSLCs are analyzed and applied in TO process to reduce the influence of deformed MSLCs on the performance of final structure. The infill resolution of MSLC within a given design domain is determined according to the minimal printable wall thickness of MSLC and structural stiffness. Both numerical and physical experimental results demonstrate the effectiveness of the proposed method.

We introduce a non-local graph attention network (NLGAT), which generates a novel global descriptor through two sub-networks for robust 3D shape classification. In the first sub-network, we capture the global relationships between points (i.e., point-point features) by designing a global relationship network (GRN). In the second sub-network, we enhance the local features with a geometric shape attention map obtained from a global structure network (GSN). To keep rotation invariant and extract more information from sparse point clouds, all sub-networks use the Gram matrices with different dimensions as input for working with robust classification. Additionally, GRN effectively preserves the low-frequency features and improves the classification results. Experimental results on various datasets exhibit that the classification effect of the NLGAT model is better than other state-of-the-art models. Especially, in the case of sparse point clouds (64 points) with noise under arbitrary SO(3) rotation, the classification result (85.4%) of NLGAT is improved by 39.4% compared with the best development of other methods.

Classical microfacet theory suffers from energy loss on materials with high roughness due to the single bounce assumption of most microfacet models. When roughness is high, there is a large chance of multiple scattering occurring among the microfacets of the surface. Without explicitly modelling for this behaviour, rough surfaces appear darker than they should. To address this issue, we present a novel method to estimate the multiple scattering contribution from a second light bounce. Our method is inspired by Zipin’s geometric construction approach, which simplifies the calculation of the light transport inside a V-groove cavity. Our experimental results demonstrate that our method is visually pleasing, physically plausible, and artifact-free compared to recent multiple scattering works. Additionally, the low computational cost makes our model suitable for real-time rendering.

Abstract We propose to use nonlinear shape functions represented as neural networks in numerical coarsening to achieve generalization capability as well as good accuracy. To overcome the challenge of generalization to different simulation scenarios, especially nonlinear materials under large deformations, our key idea is to replace the linear mapping between coarse and fine meshes adopted in previous works with a nonlinear one represented by neural networks. However, directly applying an end‐to‐end neural representation leads to poor performance due to over‐huge parameter space as well as failing to capture some intrinsic geometry properties of shape functions. Our solution is to embed geometry constraints as the prior knowledge in learning, which greatly improves training efficiency and inference robustness. With the trained neural shape functions, we can easily adopt numerical coarsening in the simulation of various hyperelastic models without any other preprocessing step required. The experiment results demonstrate the efficiency and generalization capability of our method over previous works.

We propose a novel method to compute globally injective parameterizations with arbitrary positional constraints on disk topology meshes. Central to this method is the use of a scaffold mesh that reduces the globally injective constraint to a locally flipfree condition. Hence, given an initial parameterized mesh containing flipped triangles and satisfying the positional constraints, we only need to remove the flips of a overall mesh consisting of the parameterized mesh and the scaffold mesh while always meeting positional constraints. To successfully apply this idea, we develop two key techniques. Firstly, an initialization method is used to generate a valid scaffold mesh and mitigate difficulties in eliminating flips. Secondly, edge-based remeshing is used to optimize the regularity of the scaffold mesh containing flips, thereby improving practical robustness. Compared to state-of-the-art methods, our method is much more robust. We demonstrate the capability and feasibility of our method on a large number of complex meshes.

Unsupervised domain adaptation is an appealing technique to learn robust classifiers for unlabeled target domain by borrowing knowledge from well-established source domain. However, previous works mainly suffer from two limitations: 1) the classifier trained on labeled source data may be prone to overfitting the source distribution, lowering its performance on the target domain; 2) the adaptation process will be misled by conditional distribution matching using hard pseudo labels of target samples. This paper presents a Dual-Level Adaptive and Discriminative (DLAD) classifier learning framework, in which transfer classifier and distribution adaptation can be mutually beneficial for effective knowledge transfer. Specifically, we aim to achieve a domain-level adaptive classifier by considering structural risk minimization (SRM) on both domains and performing weighted distribution adaptation, which facilitates joint classifier learning in a semi-supervised manner. To further achieve a class-level discriminative classifier, we explicitly leverage unlabeled target data to promote classifier learning based on class probabilities, which refines the decision boundary to be more discriminative for unlabeled target data. To the best of our knowledge, DLAD is the first attempt to consider the principle of SRM on the target domain, which significantly boosts the discriminative power of transfer classifier and yields a tighter generalization bound. Experimental evaluations on several standard cross-domain datasets show that DLAD significantly outperforms other competitive methods.

Autoscanning of an unknown environment is the key to many AR/VR and robotic applications. However, autonomous reconstruction with both high efficiency and quality remains a challenging problem. In this work, we propose a reconstruction-oriented autoscanning approach, called ScanBot, which utilizes hierarchical deep reinforcement learning techniques for global region-of-interest (ROI) planning to improve the scanning efficiency and local next-best-view (NBV) planning to enhance the reconstruction quality. Given the partially reconstructed scene, the global policy designates an ROI with insufficient exploration or reconstruction. The local policy is then applied to refine the reconstruction quality of objects in this region by planning and scanning a series of NBVs. A novel mixed 2D-3D representation is designed for these policies, where a 2D quality map with tailored quality channels encoding the scanning progress is consumed by the global policy, and a coarse-to-fine 3D volumetric representation that embodies both local environment and object completeness is fed to the local policy. These two policies iterate until the whole scene has been completely explored and scanned. To speed up the learning of complex environmental dynamics and enhance the agent's memory for spatial-temporal inference, we further introduce two novel auxiliary learning tasks to guide the training of our global policy. Thorough evaluations and comparisons are carried out to show the feasibility of our proposed approach and its advantages over previous methods. Code and data are available at https://github.com/HezhiCao/Scanbot.

We propose an efficient method to construct sparse cone singularities under distortion-bounded constraints for conformal parameterizations. Central to our algorithm is using the technique of shape derivatives to move cones for distortion reduction without changing the number of cones. In particular, the supernodal sparse Cholesky update significantly accelerates this movement process. To satisfy the distortion-bounded constraint, we alternately move cones and add cones. The capability and feasibility of our approach are demonstrated over a data set containing 3885 models. Compared with the state-of-the-art method, we achieve an average acceleration of 15 times and slightly fewer cones for the same amount of distortion.

Autonomous surface reconstruction of 3D scenes has been intensely studied in recent years, however, it is still difficult to accurately reconstruct all the surface details of complex scenes with complicated object relations and severe occlusions, which makes the reconstruction results not suitable for direct use in applications such as gaming and virtual reality. Therefore, instead of reconstructing the detailed surfaces, we aim to recompose the scene with CAD models retrieved from a given dataset to faithfully reflect the object geometry and arrangement in the given scene. Moreover, unlike most of the previous works on scene CAD recomposition requiring an offline reconstructed scene or captured video as input, which leads to significant data redundancy, we propose a novel online scene CAD recomposition method with autonomous scanning, which efficiently recomposes the scene with the guidance of automatically optimized Next-Best-View (NBV) in a single online scanning pass. Based on the key observation that spatial relation in the scene can not only constrain the object pose and layout optimization but also guide the NBV generation, our system consists of two key modules: relation-guided CAD recomposition module that uses relation-constrained global optimization to get accurate object pose and layout estimation, and relation-aware NBV generation module that makes the exploration during the autonomous scanning tailored for our composition task. Extensive experiments have been conducted to show the superiority of our method over previous methods in scanning efficiency and retrieval accuracy as well as the importance of each key component of our method.

Differentiable physics simulation has shown its efficacy in inverse design problems. Given the pervasiveness of the diverse interactions between fluids and solids in life, a differentiable simulator for the inverse design of the motion of rigid objects in two-way fluid-rigid coupling is also demanded. There are two main challenges to develop a differentiable two-way fluid-solid coupling simulator for rigid body control tasks: the ubiquitous, discontinuous contacts in fluid-solid interactions, and the high computational cost of gradient formulation due to the large number of degrees of freedom (DoF) of fluid dynamics. In this work, we propose a novel differentiable SPH-based two-way fluid-rigid coupling simulator to address these challenges. Our purpose is to provide a differentiable simulator for SPH which incorporates a unified representation for both fluids and solids using particles. However, naively differentiating the forward simulation of the particle system encounters gradient explosion issues. We investigate the instability in differentiating the SPH-based fluid-rigid coupling simulator and present a feasible gradient computation scheme to address its differentiability. In addition, we also propose an efficient method to compute the gradient of fluid-rigid coupling without incurring the high computational cost of differentiating the entire high-DoF fluid system. We show the efficacy, scalability, and extensibility of our method in various challenging rigid body control tasks with diverse fluid-rigid interactions and multi-rigid contacts, achieving up to an order of magnitude speedup in optimization compared to baseline methods in experiments.

The 3D visual optical arts create fascinating special effects by carefully designing interactions between objects and light sources. One of the essential types is 3D reflection art, which aims to create reflectors that can display different images when viewed from different directions. Existing works produce impressive visual effects. Unfortunately, previous works discretize the reflector surface with regular grids/facets, leading to a large parameter space and a high optimization time cost. In this paper, we introduce a new type of 3D reflection art -scratch-based reflection art, which allows for a more compact parameter space, easier fabrication, and computationally efficient optimization. To design a 3D reflection art with scratches, we formulate it as a multi-view optimization problem and introduce differentiable rendering to enable efficient gradient-based optimizers. For that, we propose an analytical scratch rendering approach, together with a high-performance rendering pipeline, allowing efficient differentiable rendering. As a consequence, we could display multiple images on a single metallic board with only several minutes for optimization. We demonstrate our work by showing virtual objects and manufacturing our designed reflectors with a carving machine.

We present ETER, an elastic tessellation framework for rendering large-scale NURBS models with pixel-accurate and crack-free quality at real-time frame rates. We propose a highly parallel adaptive tessellation algorithm to achieve pixel accuracy, measured by the screen space error between the exact surface and its triangulation. To resolve a bottleneck in NURBS rendering, we present a novel evaluation method based on uniform sampling grids and accelerated by GPU Tensor Cores. Compared to evaluation based on hardware tessellation, our method has achieved a significant speedup of 2.9 to 16.2 times depending on the degrees of the patches. We develop an efficient crack-filling algorithm based on conservative rasterization and visibility buffer to fill the tessellation-induced cracks while greatly reducing the jagged effect introduced by conservative rasterization. We integrate all our novel algorithms, implemented in CUDA, into a GPU NURBS rendering pipeline based on Mesh Shaders and hybrid software/hardware rasterization. Our performance data on a commodity GPU show that the rendering pipeline based on ETER is capable of rendering up to 3.7 million patches (0.25 billion tessellated triangles) in real-time (30FPS). With its advantages in performance, scalability, and visual quality in rendering large-scale NURBS models, a real-time tessellation solution based on ETER can be a powerful alternative or even a potential replacement for the existing pre-tessellation solution in CAD systems.

We propose a novel method to compute high-quality piecewise developable approximations for triangular meshes. Central to our approach is an evolutionary genetic algorithm for optimizing the combinatorial and discontinuous fitness function, including the approximation error, the number of patches, the patch boundary length, and the penalty for small patches and narrow regions within patches. The genetic algorithm's operations (i.e., initialization, selection, mutation, and crossover) are explicitly designed to minimize the fitness function. The main challenge is evaluating the fitness function's approximation error as it requires developable patches, which are difficult or time-consuming to obtain. Resolving the challenge is based on a critical observation: the approximation error and the mapping distortion between an input surface and its developable approximation are positively correlated empirically. To efficiently measure distortion without explicitly generating developable shapes, we creatively use conformal mapping techniques. Then, we control the mapping distortion at a relatively low level to achieve high shape similarity in the genetic algorithm. The feasibility and effectiveness of our method are demonstrated over 240 complex examples. Compared with the state-of-the-art methods, our results have much smaller approximation errors, fewer patches, shorter patch boundaries, and fewer small patches and narrow regions.

Many geometric optimization problems contain manifold constraints that restrict the optimized vertices on some specified manifold surface. The constraints are highly nonlinear and non-convex, therefore existing methods usually suffer from a breach of condition or low optimization quality. In this article, we present a novel divide-and-conquer methodology for manifold-constrained geometric optimization problems. Central to our methodology is to use local parameterizations to decouple the optimization with hard constraints, which transforms nonlinear constraints into linear constraints. We decompose the input mesh into a set of developable or nearly-developable overlapping patches with disc topology, then flatten each patch into the planar domain with very low isometric distortion, optimize vertices with linear constraints and recover the patch. Finally, we project it onto the constrained manifold surface. We demonstrate the applicability and robustness of our methodology through a variety of geometric optimization tasks. Experimental results show that our method performs much better than existing methods.

Reflectional symmetry is a ubiquitous pattern in nature. Previous works usually solve this problem by voting or sampling, suffering from high computational cost and randomness. In this article, we propose a learning-based approach to intrinsic reflectional symmetry detection. Instead of directly finding symmetric point pairs, we parametrize this self-isometry using a functional map matrix, which can be easily computed given the signs of Laplacian eigenfunctions under the symmetric mapping. Therefore, we manually label the eigenfunction signs for a variety of shapes and train a novel neural network to predict the sign of each eigenfunction under symmetry. Our network aims at learning the global property of functions and consequently converts the problem defined on the manifold to the functional domain. By disentangling the prediction of the matrix into separated bases, our method generalizes well to new shapes and is invariant under perturbation of eigenfunctions. Through extensive experiments, we demonstrate the robustness of our method in challenging cases, including different topology and incomplete shapes with holes. By avoiding random sampling, our learning-based algorithm is over 20 times faster than state-of-the-art methods, and meanwhile, is more robust, achieving higher correspondence accuracy in commonly used metrics.

In this paper, we propose HeadNeRF, a novel NeRF-based parametric head model that integrates the neural radiance field to the parametric representation of the human head. It can render high fidelity head images in real-time on modern GPUs, and supports directly controlling the generated images' rendering pose and various semantic attributes. Different from existing related parametric models, we use the neural radiance fields as a novel 3D proxy instead of the traditional 3D textured mesh, which makes that HeadNeRF is able to generate high fidelity images. However, the computationally expensive rendering process of the original NeRF hinders the construction of the parametric NeRF model. To address this issue, we adopt the strategy of integrating 2D neural rendering to the rendering process of NeRF and design novel loss terms. As a result, the rendering speed of HeadNeRF can be significantly accelerated, and the rendering time of one frame is reduced from 5s to 25ms. The well designed loss terms also improve the rendering accuracy, and the fine-level details of the human head, such as the gaps between teeth, wrinkles, and beards, can be represented and synthesized by HeadNeRF. Extensive experimental results and several applications demonstrate its effectiveness. The trained parametric model is available at https://github.com/CrisHY1995/headnerf.

Abstract We propose an interactive method to edit a discrete Chebyshev net, which is a quad mesh with edges of the same length. To ensure that the edited mesh is always a discrete Chebyshev net, the maximum difference of all edge lengths should be zero during the editing process. Hence, we formulate an objective function using ℓp‐norm (p > 2) to force the maximum length deviation to approach zero in practice. To optimize the nonlinear and non‐convex objective function interactively and efficiently, we develop a novel second‐order solver. The core of the solver is to construct a new convex majorizer for our objective function to achieve fast convergence. We present two acceleration strategies to further reduce the optimization time, including adaptive p change and adaptive variables reduction. A large number of experiments demonstrate the capability and feasibility of our method for interactively editing complex discrete Chebyshev nets.

Abstract We propose a novel method to generate a high‐order triangular mesh for an input 2D domain with two key characteristics: (1) the mesh precisely conforms to a set of input piecewise rational domain curves, and (2) the geometric map on each curved triangle is injective. Central to the algorithm is a new sufficient condition for placing control points of a rational Bézier triangle to guarantee that the conformance and injectivity constraints are theoretically satisfied. Taking advantage of this condition, we provide an explicit construct that robustly creates higher‐order 2D meshes satisfying the two characteristics. We demonstrate the robustness and effectiveness of our algorithm over a data set containing 2200 examples.

Abstract We propose a simple yet effective method to perform surface remeshing with hard constraints, such as bounding approximation errors and ensuring Delaunay conditions. The remeshing is formulated as a constrained optimization problem, where the variables contain the mesh connectivity and the mesh geometry. To solve it effectively, we adopt traditional local operations, including edge split, edge collapse, edge flip, and vertex relocation, to update the variables. Central to our method is an evolutionary vertex optimization algorithm, which is derivative‐free and robust. The feasibility and practicability of our method are demonstrated in two applications, including error‐bounded Delaunay mesh simplification and error‐bounded angle improvement with a given number of vertices, over many models. Compared to state‐of‐the‐art methods, our method achieves higher remeshing quality.

Abstract We propose a novel topology optimization method to efficiently minimize the maximum compliance for a high‐resolution model bearing uncertain external loads. Central to this approach is a modified power method that can quickly compute the maximum eigenvalue to evaluate the worst‐case compliance, enabling our method to be suitable for large‐scale topology optimization. After obtaining the worst‐case compliance, we use the adjoint variable method to perform the sensitivity analysis for updating the density variables. By iteratively computing the worst‐case compliance, performing the sensitivity analysis, and updating the density variables, our algorithm achieves the optimized models with high efficiency. The capability and feasibility of our approach are demonstrated over various large‐scale models. Typically, for a model of size 512×170×170 and 69934 loading nodes, our method took about 50 minutes on a desktop computer with an NVIDIA GTX 1080Ti graphics card with 11 GB memory.

Detecting points of interest on 3D shapes is a fundamental research problem in geometry processing. Due to the complicated relationship between points of interest and their geometric features, detecting points of interest on any given 3D shape remains challenging. Due to the lack of training data, previous data-driven methods for detecting 3D points of interest mainly focus on utilizing hand-crafted geometric features to predict the probabilities of each point being a POI, which greatly limits detection performance. In this paper, we propose a novel algorithm for detecting 3D points of interest by using projective neural networks. Our method first projects the labeled training 3D shapes into multiple 2D views and then learns the required features from the 2D views in an end-to-end fashion. The points of interest on test 3D shapes are then automatically detected by applying the learned neural network and our improved density peak clustering. Our method relies neither on hand-crafted feature descriptors nor a large quantity of expensive 3D training data to obtain satisfactory results. Experimental results show significantly superior detection performance of our method over the state-of-the-art methods.

Image smoothing is a prerequisite for many computer vision and graphics applications. In this article, we raise an intriguing question whether a dataset that semantically describes meaningful structures and unimportant details can facilitate a deep learning model to smooth complex natural images. To answer it, we generate ground-truth labels from easy samples by candidate generation and a screening test and synthesize hard samples in structure-preserving smoothing by blending intricate and multifarious details with the labels. To take full advantage of this dataset, we present a joint edge detection and structure-preserving image smoothing neural network (JESS-Net). Moreover, we propose the distinctive total variation loss as prior knowledge to narrow the gap between synthetic and real data. Experiments on different datasets and real images show clear improvements of our method over the state of the arts in terms of both the image cleanness and structure-preserving ability. Code and dataset are available at https://github.com/YidFeng/Easy2Hard.

Exact 3D path generation is a fundamental problem of designing a mechanism to make a point exactly move along a prescribed 3D path , driven by a single actuator. Existing mechanisms are insufficient to address this problem. Planar linkages and their combinations with gears and/or plate cams can only generate 2D paths while 1-DOF spatial linkages can only generate 3D paths with rather simple shapes. In this paper, we present a new 3D cam-linkage mechanism, consisting of two 3D cams and five links, for exactly generating a continuous 3D path. To design a 3D cam-linkage mechanism, we first model a 3-DOF 5-bar spatial linkage to exactly generate a prescribed 3D path and then reduce the spatial linkage's DOFs from 3 to 1 by composing the linkage with two 3D cam-follower mechanisms. Our computational approach optimizes the 3D cam-linkage mechanism's topology and geometry to minimize the mechanism's total weight while ensuring smooth, collision-free, and singularity-free motion. We show that our 3D cam-linkage mechanism is able to exactly generate a continuous 3D path with arbitrary shape and a finite number of C 0 points, evaluate the mechanism's kinematic performance with 3D printed prototypes, and demonstrate that the mechanism can be generalized for exact 3D motion generation.

When conducting autonomous scanning for the online reconstruction of unknown indoor environments, robots have to be competent at exploring scene structure and reconstructing objects with high quality. Our key observation is that different tasks demand specialized scanning properties of robots: rapid moving speed and far vision for global exploration and slow moving speed and narrow vision for local object reconstruction, which are referred as two different scanning modes: explorer and reconstructor , respectively. When requiring multiple robots to collaborate for efficient exploration and fine-grained reconstruction, the questions on when to generate and how to assign those tasks should be carefully answered. Therefore, we propose a novel asynchronous collaborative autoscanning method with mode switching, which generates two kinds of scanning tasks with associated scanning modes, i.e., exploration task with explorer mode and reconstruction task with reconstructor mode, and assign them to the robots to execute in an asynchronous collaborative manner to highly boost the scanning efficiency and reconstruction quality. The task assignment is optimized by solving a modified Multi-Depot Multiple Traveling Salesman Problem (MDMTSP). Moreover, to further enhance the collaboration and increase the efficiency, we propose a task-flow model that actives the task generation and assignment process immediately when any of the robots finish all its tasks with no need to wait for all other robots to complete the tasks assigned in the previous iteration. Extensive experiments have been conducted to show the importance of each key component of our method and the superiority over previous methods in scanning efficiency and reconstruction quality.

We propose a novel method to generate sparse integer-constrained cone singularities with low distortion constraints for conformal parameterizations. Inspired by [Fang et al. 2021; Soliman et al. 2018], the cone computation is formulated as a constrained optimization problem, where the objective is the number of cones measured by the ℓ 0 -norm of Gaussian curvature of vertices, and the constraint is to restrict the cone angles to be multiples of π /2 and control the distortion while ensuring that the Yamabe equation holds. Besides, the holonomy angles for the non-contractible homology loops are additionally required to be multiples of π /2 for achieving rotationally seamless conformal parameterizations. The Douglas-Rachford (DR) splitting algorithm is used to solve this challenging optimization problem, and our success relies on two key components. First, replacing each integer constraint with the intersection of a box set and a sphere enables us to manage the subproblems in DR splitting update steps in the continuous domain. Second, a novel solver is developed to optimize the ℓ 0 -norm without any approximation. We demonstrate the effectiveness and feasibility of our algorithm on a data set containing 3885 models. Compared to state-of-the-art methods, our method achieves a better tradeoff between the number of cones and the parameterization distortion.

Floorplan generation has drawn widespread interest in the community. Recent learning-based methods for generating realistic floorplans have made significant progress while a complex heuristic post-processing is still necessary to obtain desired results. In this paper, we propose a novel wall-oriented method, called WallPlan , for automatically and efficiently generating plausible floorplans from various design constraints. We pioneer the representation of the floorplan as a wall graph with room labels and consider the floorplan generation as a graph generation. Given the boundary as input, we first initialize the boundary with windows predicted by WinNet. Then a graph generation network GraphNet and semantics prediction network LabelNet are coupled to generate the wall graph progressively by imitating graph traversal. WallPlan can be applied for practical architectural designs, especially the wall-based constraints. We conduct ablation experiments, qualitative evaluations, quantitative comparisons, and perceptual studies to evaluate our method's feasibility, efficacy, and versatility. Intensive experiments demonstrate our method requires no post-processing, producing higher quality floorplans than state-of-the-art techniques.

In the mirror cup and saucer art created by artists Yul Cho and Sang-Ha Cho, part of the saucer is directly visible to the viewer, while the other part of the saucer is occluded and can only be seen as a reflection through a mirror cup. Thus, viewers see an image directly on the saucer and another image on the mirror cup; however, the existing art design is limited to wavelike saucers. In this work, we propose a general computational framework for mirror cup and saucer art design. As input, we take from the user one image for the direct view, one image for the reflected view, and the base shape of the saucer. Our algorithm then generates a suitable saucer shape by deforming the input shape. We formulate this problem as a constrained optimization for the saucer surface. Our framework solves for the fine geometry details on the base shape along with its texture, such that when a mirror cup is placed on the saucer, the user-specified images are observed as direct and reflected views. Through extensive experiments, we demonstrate the effectiveness of our framework and the great design flexibility that it offers to users. We further validate the produced art pieces by fabricating the colored saucers using three-dimensional printing.

We propose a novel method to compute a piecewise mesh with a few developable patches and a small approximation error for an input triangular mesh. Our key observation is that a deformed mesh after enforcing discrete developability is easily partitioned into nearly developable patches. To obtain the nearly developable mesh, we present a new edge-oriented notion of discrete developability to define a developability-encouraged deformation energy, which is further optimized by the block nonlinear Gauss-Seidel method. The key to successfully applying this optimizer is three types of auxiliary variables. Then, a coarse-to-fine segmentation technique is developed to partition the deformed mesh into a small set of nearly discrete developable patches. Finally, we refine the segmented mesh to reduce the discrete Gaussian curvature while keeping the patches smooth and the approximation error small. In practice, our algorithm achieves a favorable tradeoff between the number of developable patches and the approximation error. We demonstrate the feasibility and practicability of our method over various examples, including seventeen physical manufacturing models with paper.

In the game and film industries, modeling 3D heads plays a very important role in designing characters. Although human head modeling has been researched for a long time, few works have focused on animal-like heads, which are of more diverse shapes and richer geometric details. In this article, we present SAniHead, an interactive system for creating animal-like heads with a mesh representation from dual-view sketches. Our core technical contribution is a view-surface collaborative mesh generative network. Initially, a graph convolutional neural network (GCNN) is trained to learn the deformation of a template mesh to fit the shape of sketches, giving rise to a coarse model. It is then projected into vertex maps where image-to-image translation networks are performed for detail inference. After back-projecting the inferred details onto the meshed surface, a new GCNN is trained for further detail refinement. The modules of view-based detail inference and surface-based detail refinement are conducted in an alternating cascaded fashion, collaboratively improving the model. A refinement sketching interface is also implemented to support direct mesh manipulation. Experimental results show the superiority of our approach and the usability of our interactive system. Our work also contributes a 3D animal head dataset with corresponding line drawings.

In this approach, we present an efficient topology and geometry optimization of triply periodic minimal surfaces (TPMS) based porous shell structures, which can be represented, analyzed, optimized and stored directly using functions. The proposed framework is directly executed on functions instead of remeshing (tetrahedral/hexahedral), and this framework substantially improves the controllability and efficiency. Specifically, a valid TPMS-based porous shell structure is first constructed by function expressions. The porous shell permits continuous and smooth changes of geometry (shell thickness) and topology (porous period). The porous structures also inherit several of the advantageous properties of TPMS, such as smoothness, full connectivity (no closed hollows), and high controllability. Then, the problem of filling an object's interior region with porous shell can be formulated into a constraint optimization problem with two control parameter functions. Finally, an efficient topology and geometry optimization scheme is presented to obtain optimized scale-varying porous shell structures. In contrast to traditional heuristic methods for TPMS, our work directly optimize both the topology and geometry of TPMS-based structures. Various experiments have shown that our proposed porous structures have obvious advantages in terms of efficiency and effectiveness.

Recent studies have shown that the Total Generalized Variation (TGV) is highly effective in preserving sharp features as well as smooth transition variations for image processing tasks. However, currently there is no existing work that is suitable for applying TGV to 3D data, in particular, triangular meshes. In this article, we develop a novel framework for discretizing second-order TGV on triangular meshes. Further, we propose a TGV-based variational method for the denoising of face normal fields on triangular meshes. The TGV regularizer in our method is composed of a first-order term and a second-order term, which are automatically balanced. The first-order term allows our TGV regularizer to locate and preserve sharp features, while the second-order term allows our regularizer to recognize and recover smoothly curved regions. To solve the optimization problem, we introduce an efficient iterative algorithm based on variable-splitting and augmented Lagrangian method. Extensive results and comparisons on synthetic and real scanning data validate that the proposed method outperforms the state-of-the-art visually and numerically.

The emerging 4D printing techniques open new horizons for fabricating self-actuated deformable objects by combing strength of 3D printing and stimuli-responsive shape memory materials. This article focuses on designing self-actuated deformable solids for 4D printing such that a solid can be programmed into a temporary shape and later recovers to its original shape after heating. To avoid a high material cost, we choose a dual-material strategy that mixes an expensive thermo-responsive shape memory polymer (SMP) material with a common elastic material, which however leads to undesired deformation at the shape programming stage. We model this shape programming process as two elastic models with different parameters linked by a median shape based on customizing a constitutive model of thermo-responsive SMPs. Taking this material modeling as a foundation, we formulate our design problem as a nonconvex optimization to find the distribution of SMP materials over the whole object as well as the median shape, and develop an efficient and parallelizable method to solve it. We show that our proposed approach is able to design self-actuated deformable objects that cannot be achieved by state of the art approaches, and demonstrate their usefulness with three example applications.

The design of the light-weight infill structure is a hot research topic in additive manufacturing. In recent years, various infill structures have been proposed to reduce the amount of printing material. However, 3D models filled with them may have very different structural performances under different loading conditions. In addition, most of them are not self-supporting. To mitigate these issues, a novel light-weight infill structure based on the layer construction is proposed in this article. The layers of the proposed infill structure continuously and periodically transform between triangles and hexagons. The geometries of two adjacent layers are controlled to be self-supporting for different 3D printing technologies. The machine code (Gcode) of the filled 3D model is generated in the construction of the infill structure for 3D printers. That means 3D models filled with the proposed infill structure do not need an extra slicing process before printing, which is time consuming in some cases. Structural simulations and physical experiments demonstrate that our infill structure has comparable structural performance under different loading conditions. Furthermore, the relationship between the structural stiffness and the parameters of the infill structure is investigated, which will be helpful for non-professional users.

In this paper, we propose StereoPIFu, which integrates the geometric constraints of stereo vision with implicit function representation of PIFu, to recover the 3D shape of the clothed human from a pair of low-cost rectified images. First, we introduce the effective voxel-aligned features from a stereo vision-based network to enable depth-aware reconstruction. Moreover, the novel relative z-offset is employed to associate predicted high-fidelity human depth and occupancy inference, which helps restore fine-level surface de-tails. Second, a network structure that fully utilizes the geometry information from the stereo images is designed to improve the human body reconstruction quality. Consequently, our StereoPIFu can naturally infer the human body’s spatial location in camera space and maintain the correct relative position of different parts of the human body, which enables our method to capture human performance. Compared with previous works, our StereoPIFu significantly improves the robustness, completeness, and accuracy of the clothed human reconstruction, which is demonstrated by extensive experimental results.

Abstract Modeling 3D objects on existing software usually requires a heavy amount of interactions, especially for users who lack basic knowledge of 3D geometry. Sketch‐based modeling is a solution to ease the modelling procedure and thus has been researched for decades. However, modelling a man‐made shape with complex structures remains challenging. Existing methods adopt advanced deep learning techniques to map holistic sketches to 3D shapes. They are still bottlenecked to deal with complicated topologies. In this paper, we decouple the task of sketch2shape into a part generation module and a part assembling module, where deep learning methods are leveraged for the implementation of both modules. By changing the focus from holistic shapes to individual parts, it eases the learning process of the shape generator and guarantees high‐quality outputs. With the learned automated part assembler, users only need a little manual tuning to obtain a desired layout. Extensive experiments and user studies demonstrate the usefulness of our proposed system.

Abstract Path tracing has been used for real‐time renderings, thanks to the powerful GPU device. Unfortunately, path tracing produces noisy rendered results, thus, filtering or denoising is often applied as a post‐process to remove the noise. Previous works produce high‐quality denoised results, by accumulating the temporal samples. However, they cannot handle the details from bidirectional reflectance distribution function (BRDF) maps (e.g. roughness map). In this paper, we introduce the BRDF pre‐integration factorization for denoising to better preserve the details from BRDF maps. More specifically, we reformulate the rendering equation into two components: the BRDF pre‐integration component and the weighted‐lighting component. The BRDF pre‐integration component is noise‐free, since it does not depend on the lighting. Another key observation is that the weighted‐lighting component tends to be smooth and low‐frequency, which indicates that it is more suitable for denoising than the final rendered image. Hence, the weighted‐lighting component is denoised individually. Our BRDF pre‐integration demodulation approach is flexible for many real‐time filtering methods. We have implemented it in spatio‐temporal variance‐guided filtering (SVGF), ReLAX and ReBLUR. Compared to the original methods, our method manages to better preserve the details from BRDF maps, while both the memory and time cost are negligible.

A geometric mapping establishes a correspondence between two domains. Since no real object has zero or negative volume, such a mapping is required to be inversion-free. Computing inversion-free mappings is a fundamental task in numerous computer graphics and geometric processing applications, such as deformation, texture mapping, mesh generation, and others. This task is usually formulated as a non-convex, nonlinear, constrained optimization problem. Various methods have been developed to solve this optimization problem. As well as being inversion-free, different applications have various further requirements. We expand the discussion in two directions to (i) problems imposing specific constraints and (ii) combinatorial problems. This report provides a systematic overview of inversion-free mapping construction, a detailed discussion of the construction methods, including their strengths and weaknesses, and a description of open problems in this research field.

Motion control, both on the trajectory and timing, is crucial for mechanical automata to perform functionalities such as walking and entertaining. We present composite cam-follower mechanisms that can control their spatial-temporal motions to exactly follow trajectories and timings specified by users, and propose a computational technique to model, design, and optimize these mechanisms. The building blocks of our mechanisms are a new kind of cam-follower mechanism with a modified joint, in which the follower can perform spatial motion on a planar, cylindrical, or spherical surface controlled by the 3D cam's profile. We parameterize the geometry of these cam-follower mechanisms, formulate analytical equations to model their kinematics and dynamics, and present a method to combine multiple cam-follower mechanisms into a working mechanism. Taking this modeling as a foundation, we propose a computational approach to designing and optimizing the geometry and layout of composite cam-follower mechanisms, with an objective of performing target spatial-temporal motions driving by a small motor torque. We demonstrate the effectiveness of our technique by designing different kinds of personalized automata and showing results not attainable by conventional mechanisms.

In virtual reality (VR), the virtual scenes are pre-designed by creators. Our physical surroundings, however, comprise significantly varied sizes, layouts, and components. To bridge the gap and further enable natural navigation, recent solutions have been proposed to redirect users or recreate the virtual content. However, they suffer from either interrupted experience or distorted appearances. We present a novel VR-oriented algorithm that automatically restructures a given virtual scene for a user’s physical environment. Different from the previous methods, we introduce neither interrupted walking experience nor curved appearances. Instead, a perception-aware function optimizes our retargeting technique to preserve the fidelity of the virtual scene that appears in VR head-mounted displays. Besides geometric and topological properties, it emphasizes the unique first-person view perceptual factors in VR, such as dynamic visibility and objectwise relationships. We conduct both analytical experiments and subjective studies. The results demonstrate our system’s versatile capability and practicability for natural navigation in VR: It reduces the virtual space by 40% without statistical loss of perceptual identicality.

We propose a novel method to generate sparse cone singularities with bounded distortion constraints for conformal parameterizations. It is formulated as minimizing the ℓ 0 -norm of Gaussian curvature of vertices with hard constraints of bounding the distortion that is measured by the ℓ 2 -norm of the log conformal factor. We use the reweighted ℓ 1 -norm to approximate the ℓ 0 -norm and solve each convex weighted ℓ 1 minimization subproblem by the Douglas-Rachford (DR) splitting scheme. To quickly generate sparse cones, we modify DR splitting by weighting the ℓ 2 -norm of the proximal mapping to force the small Gaussian curvature to quickly approach zero. Accordingly, compared with the conventional DR splitting, the modified method performs one to two orders of magnitude faster. Besides, we perform variable substitution of log conformal factors to simplify the computation process for acceleration. Our algorithm is able to bound distortion to compute sparse cone singularities, so that the resulting conformal parameterizations achieve a favorable tradeoff between the area distortion and the number of cones. We demonstrate its effectiveness and feasibility on a large number of models.

We present a highly efficient method for interactive volumetric meshless shape deformation. Our method operates within a low dimensional sub-space of shape-aware C ∞ harmonic maps, and is the first method that is guaranteed to produce a smooth locally injective deformation in 3D. Unlike mesh-based methods in which local injectivity is enforced on tetrahedral elements, our method enforces injectivity on a sparse set of domain samples. The main difficulty is then to certify the map as locally injective throughout the entire domain. This is done by utilizing the Lipschitz continuity property of the harmonic basis functions. We show a surprising relation between the Lipschitz constant of the smallest singular value of the map Jacobian and the norm of the Hessian. We further carefully derive a Lipschitz constant for the Hessian, and develop a sufficient condition for the injectivity certification. This is done by utilizing the special structure of the harmonic basis functions combined with a novel regularization term that pushes the Lipschitz constants further down. As a result, the injectivity analysis can be performed on a relatively sparse set of samples. Combined with a parallel GPU-based implementation, our method can produce superior deformations with unique quality guarantees at real-time rates which were possible only in 2D so far.

We propose a novel method to model and fabricate shapes using a small set of specified discrete equivalence classes of triangles. The core of our modeling technique is a fabrication-error-driven remeshing algorithm. Given a triangle and a template triangle, which are coplanar and have one-to-one corresponding vertices, we define their similarity error from a manufacturing point of view as follows: the minimizer of the maximum of the three distances between the corresponding pair of vertices concerning a rigid transformation. To compute the similarity error, we convert it into an easy-to-compute form. Then, a greedy remeshing method is developed to optimize the topology and geometry of the input mesh to minimize the fabrication error defined as the maximum similarity error of all triangles. Besides, constraints are enforced to ensure the similarity between input and output shapes and the smoothness of the resulting shapes. Since the fabrication error has been considered during the modeling process, the fabrication process is easy to proceed. To assist users in performing fabrication using common materials and tools manually, we present a straightforward manufacturing solution. The feasibility and practicability of our method are demonstrated over various examples, including seven physical manufacturing models with only nine template triangles.

Low isometric distortion is often required for mesh parameterizations. A configuration of some vertices, where the distortion is concentrated, provides a way to mitigate isometric distortion, but determining the number and placement of these vertices is non-trivial. We call these vertices distortion points. We present a novel and automatic method to detect distortion points using a voting strategy. Our method integrates two components: candidate generation and candidate voting. Given a closed triangular mesh, we generate candidate distortion points by executing a three-step procedure repeatedly: (1) randomly cut an input to a disk topology; (2) compute a low conformal distortion parameterization; and (3) detect the distortion points. Finally, we count the candidate points and generate the final distortion points by voting. We demonstrate that our algorithm succeeds when employed on various closed meshes with a genus of zero or higher. The distortion points generated by our method are utilized in three applications, including planar parameterization, semi-automatic landmark correspondence, and isotropic remeshing. Compared to other state-of-the-art methods, our method demonstrates stronger practical robustness in distortion point detection.

With the advent of deep neural networks, learning-based single-view reconstruction has gained popularity. However, in 3D, there is no absolutely dominant representation that is both computationally efficient and accurate yet allows for reconstructing high-resolution geometry of arbitrary topology. After all, the accurate implicit methods are time-consuming due to dense sampling and inference, while volumetric approaches are fast but limited to heavy memory usage and low accuracy. In this paper, we propose VIPNet, an end-to-end hybrid representation learning for fast and accurate single-view reconstruction under sparse implicit point guidance. Given an image, it first generates a volumetric result. Meanwhile, a corresponding implicit shape representation is learned. To balance the efficiency and accuracy, we adopt PointGenNet to learn some representative points for guiding the voxel refinement with the corresponding sparse implicit inference. A strategy of patch-based synthesis with global-local features under implicit guidance is also applied for reducing memory consumption required to generate high-resolution output. Extensive experiments demonstrate the effectiveness of our method both qualitatively and quantitatively, which indicates that our proposed hybrid learning outperforms separate representation learning. Specifically, our network not only runs 60 times faster than implicit methods but also contributes to accuracy gains. We hope it will inspire a re-thinking of hybrid representation learning.

We introduce FPConv, a novel surface-style convolution operator designed for 3D point cloud analysis. Unlike previous methods, FPConv doesn't require transforming to intermediate representation like 3D grid or graph and directly works on surface geometry of point cloud. To be more specific, for each point, FPConv performs a local flattening by automatically learning a weight map to softly project surrounding points onto a 2D grid. Regular 2D convolution can thus be applied for efficient feature learning. FPConv can be easily integrated into various network architectures for tasks like 3D object classification and 3D scene segmentation, and achieve comparable performance with existing volumetric-type convolutions. More importantly, our experiments also show that FPConv can be a complementary of volumetric convolutions and jointly training them can further boost overall performance into state-of-the-art results.

In this paper, we present a deep learning approach for shading decomposition from single garment images with complex textured albedos. We propose a multi-stream generator to infer albedo, shading, normal and lighting in a unified network and train it in a novel local-global adversarial learning framework. To support the training, we carefully build a large dataset with synthetic garment photos under various lighting conditions and textured albedos. Both quantitative and qualitative evaluations, on synthetic and real images, demonstrate the superiority of our method against the state-of-the-art.

Abstract We propose a computational and practical technique to allow home users to fabricate discrete Chebyshev nets for various 3D models. The success of our method relies on two key components. The first one is a novel and simple method to approximate discrete integrable, unit‐length, and angle‐bounded frame fields, used to model discrete Chebyshev nets. Central to our field generation process is an alternating algorithm that takes turns executing one pass to enforce integrability and another pass to approach unit length while bounding angles. The second is a practical fabrication specification. The discrete Chebyshev net is first partitioned into a set of patches to facilitate manufacturing. Then, each patch is assigned a specification on pulling, bend, and fold to fit the nets. We demonstrate the capability and feasibility of our method in various complex models.

Abstract As high‐precision 3D scanners become more and more widespread, it is easy to obtain very‐large‐scale meshes containing at least millions of vertices. However, processing these very‐large‐scale meshes is still a very challenging task due to memory limitations. This paper focuses on a fundamental geometric processing task, i.e., bijective parameterization construction. To this end, we present a spline‐enhanced method to compute bijective and low distortion parameterizations for very‐large‐scale disk topology meshes. Instead of computing descent directions using the mesh vertices as variables, we estimate descent directions for each vertex by optimizing a proxy energy defined in spline spaces. Since the spline functions contain a small set of control points, it significantly decreases memory requirement. Besides, a divide‐and‐conquer method is proposed to obtain bijective initializations, and a submesh‐based optimization strategy is developed to reduce distortion further. The capability and feasibility of our method are demonstrated over various complex models. Compared to the existing methods for bijective parameterizations of very‐large‐scale meshes, our method exhibits better scalability and requires much less memory.

We propose a novel method to efficiently compute bijective parameterizations with low distortion on disk topology meshes. Our method relies on a second-order solver. To design an efficient solver, we develop two key techniques. First, we propose a coarse shell to substantially reduce the number of collision constraints that are used to guarantee overlap-free boundaries. During the optimization process, the shell ensures the Hessian matrix with a fixed nonzero structure and a low density, thereby significantly accelerating the optimization. The second is a triangle inequality-based barrier function that effectively ensures non-intersecting boundaries. Our barrier function is C ∞ inside the locally supported region and its convex second-order approximation is able to be analytically obtained. Compared to state-of-the-art methods for optimizing bijective parameterizations, our method exhibits better scalability and is about six times faster. The performance of our bijective parameterization algorithm is comparable to state-of-the-art methods of locally flip-free parameterizations. A large number of experimental results have shown the capability and feasibility of our method.

We present a novel method to construct compatible surface meshes with bounded approximation errors. Given two oriented and topologically equivalent surfaces and a sparse set of corresponding landmarks, our method contains two steps: (1) generate compatible meshes with bounded approximation errors and (2) reduce mesh complexity while ensuring that approximation errors are always bounded. Central to the first step is a parameterization-based remeshing technique, which is capable of isotropically remeshing the input surfaces to be compatible and error-bounded. By iteratively performing a novel edge-based compatible remeshing and increasing the compatible target edge lengths, the second step effectively reduces mesh complexity while explicitly maintaining compatibility, regularity, and bounding approximation errors. Tests on various pairs of complex models demonstrate the efficacy and practicability of our method for constructing high-quality compatible meshes with bounded approximation errors.

Shape segmentation is a fundamental problem in shape analysis. Previous research shows that prior knowledge helps to improve the segmentation accuracy and quality. However, completely labeling each 3D shape in a large training data set requires a heavy manual workload. In this paper, we propose a novel weakly-supervised algorithm for segmenting 3D shapes using deep learning. Our method jointly propagates information from scribbles to unlabeled faces and learns deep neural network parameters. Therefore, it does not rely on completely labeled training shapes and only needs a really simple and convenient scribble-based partially labeling process, instead of the extremely time-consuming and tedious fully labeling processes. Various experimental results demonstrate the proposed method's superior segmentation performance over the previous unsupervised approaches and comparable segmentation performance to the state-of-the-art fully supervised methods.

We present a deep reinforcement learning method of progressive view inpainting for 3D point scene completion under volume guidance, achieving high-quality scene reconstruction from only a single depth image with severe occlusion. Our approach is end-to-end, consisting of three modules: 3D scene volume reconstruction, 2D depth map inpainting, and multi-view selection for completion. Given a single depth image, our method first goes through the 3D volume branch to obtain a volumetric scene reconstruction as a guide to the next view inpainting step, which attempts to make up the missing information; the third step involves projecting the volume under the same view of the input, concatenating them to complete the current view depth, and integrating all depth into the point cloud. Since the occluded areas are unavailable, we resort to a deep Q-Network to glance around and pick the next best view for large hole completion progressively until a scene is adequately reconstructed while guaranteeing validity. All steps are learned jointly to achieve robust and consistent results. We perform qualitative and quantitative evaluations with extensive experiments on the SUNCG data, obtaining better results than the state of the art.

Abstract In this paper, we present a practically robust method for computing foldover‐free volumetric mappings with hard linear constraints. Central to this approach is a projection algorithm that monotonically and efficiently decreases the distance from the mapping to the bounded conformal distortion mapping space. After projection, the conformal distortion of the updated mapping tends to be below the given bound, thereby significantly reducing foldovers. Since it is non‐trivial to define an optimal bound, we introduce a practical conformal distortion bound generation scheme to facilitate subsequent projections. By iteratively generating conformal distortion bounds and trying to project mappings into bounded conformal distortion spaces monotonically, our algorithm achieves high‐quality foldover‐free volumetric mappings with strong practical robustness and high efficiency. Compared with existing methods, our method computes mesh‐based and meshless volumetric mappings with no prescribed conformal distortion bounds. We demonstrate the efficacy and efficiency of our method through a variety of geometric processing tasks.

Abstract Dissection puzzles require assembling a common set of pieces into multiple distinct forms. Existing works focus on creating 2D dissection puzzles that form primitive or naturalistic shapes. Unlike 2D dissection puzzles that could be supported on a tabletop surface, 3D dissection puzzles are preferable to be steady by themselves for each assembly form. In this work, we aim at computationally designing steady 3D dissection puzzles. We address this challenging problem with three key contributions. First, we take two voxelized shapes as inputs and dissect them into a common set of puzzle pieces, during which we allow slightly modifying the input shapes, preferably on their internal volume, to preserve the external appearance. Second, we formulate a formal model of generalized interlocking for connecting pieces into a steady assembly using both their geometric arrangements and friction. Third, we modify the geometry of each dissected puzzle piece based on the formal model such that each assembly form is steady accordingly. We demonstrate the effectiveness of our approach on a wide variety of shapes, compare it with the state‐of‐the‐art on 2D and 3D examples, and fabricate some of our designed puzzles to validate their steadiness.

Abstract We present a novel method to compute bijective PolyCube‐maps with low isometric distortion. Given a surface and its pre‐axis‐aligned shape that is not an exact PolyCube shape, the algorithm contains two steps: (i) construct a PolyCube shape to approximate the pre‐axis‐aligned shape; and (ii) generate a bijective, low isometric distortion mapping between the constructed PolyCube shape and the input surface. The PolyCube construction is formulated as a constrained optimization problem, where the objective is the number of corners in the constructed PolyCube, and the constraint is to bound the approximation error between the constructed PolyCube and the input pre‐axis‐aligned shape while ensuring topological validity. A novel erasing‐and‐filling solver is proposed to solve this challenging problem. Centeral to the algorithm for computing bijective PolyCube‐maps is a quad mesh optimization process that projects the constructed PolyCube onto the input surface with high‐quality quads. We demonstrate the efficacy of our algorithm on a data set containing 300 closed meshes. Compared to state‐of‐the‐art methods, our method achieves higher practical robustness and lower mapping distortion.

Abstract Scissor structures, which transform from a compact state to an expanded state, are widely used in various fields, ranging from architectural design to aerospace applications. We focus on a challenging problem that breaks through the restriction of planarity: to design scissor structures that expand from one given 3D shape to another 3D shape. To achieve this purpose, we propose a three‐step algorithm to construct a 3D scissor structure that realizes non‐uniform concentration between two different 3D curves. First, the input shapes are divided into scissor segments, which are composed of a sequence of planar scissor units based on the shape correspondence. Secondly, we compute the scissor unit geometry of each segment in a suggestive manner. Finally, the ball‐and‐socket joints with parameterized guide slits are integrated in order to connect the scissor segments, thereby completing the 3D deployment. Judging from a series of simulation and fabrication results, we demonstrate that our approach generates deployable structures for a wide range of 3D shape pairs.

We propose a novel technique to provide multiuser real walking experiences with physical interactions in virtual reality (VR) applications. In our system, multiple users walk freely while navigating a large virtual environment within a smaller physical workspace. These users can interact with other real users or physical props in the same physical locations. The key of our method is a redirected smooth mapping that incorporates the redirected walking technique to warp the input virtual scene with small bends and low distance distortion. Users possess a wide field of view to explore the mapped virtual environment while being redirected in the real workspace. To keep multiple users away from the overlaps of the mapped virtual scenes, we present an automatic collision avoidance technique based on dynamic virtual avatars. These avatars naturally appear, move, and disappear, producing as little influence as possible on users’ walking experiences. We evaluate our multiuser real walking system through formative user studies, and demonstrate the capability and practicability of our technique in two multiuser applications.

We present a novel algorithm to refine an input atlas with bounded packing efficiency. Central to this method is the use of the axis-aligned structure that converts the general polygon packing problem to a rectangle packing problem, which is easier to achieve high packing efficiency. Given a parameterized mesh with no flipped triangles, we propose a new angle-driven deformation strategy to transform it into a set of axis-aligned charts, which can be decomposed into rectangles by the motorcycle graph algorithm. Since motorcycle graphs are not unique, we select the one balancing the trade-off between the packing efficiency and chart boundary length, while maintaining bounded packing efficiency. The axis-aligned chart often contains greater distortion than the input, so we try to reduce the distortion while bounding the packing efficiency and retaining bijection. We demonstrate the efficacy of our method on a data set containing over five thousand complex models. For all models, our method is able to produce packed atlases with bounded packing efficiency; for example, when the packing efficiency bound is set to 80%, we elongate the boundary length by an average of 78.7% and increase the distortion by an average of 0.0533%. Compared to state-of-the-art methods, our method is much faster and achieves greater packing efficiency.

Some artists peel citrus fruits into a variety of elegant 2D shapes, depicting animals, plants, and cartoons. It is a creative art form, called Citrus Peeling Art. This art form follows the conservation principle, i.e., each shape must be created using one entire peel. Central to this art is finding optimal cut lines so that the citruses can be cut and unfolded into the desired shapes. However, it is extremely difficult for users to imagine and generate cuts for their desired shapes. To this end, we present a computational method for citrus peeling art designs. Our key insight is that instead of solving the difficult cut generation problem, we map a designed input shape onto a citrus in an attempt to cover the entire citrus and use the mapped boundary to generate the cut paths. Sometimes, a mapped shape is unable to completely cover a citrus. Consequently, we have developed five customized ways of interaction that are used to rectify the input shape so that it is suitable for citrus peeling art. The mapping process and user interactions are iteratively conducted to satisfy a user's design intentions. A large number of experiments, including a formative user study, demonstrate the capability and practicability of our method for peeling art design and construction.

We propose a novel data-driven technique for automatically and efficiently generating floor plans for residential buildings with given boundaries. Central to this method is a two-stage approach that imitates the human design process by locating rooms first and then walls while adapting to the input building boundary. Based on observations of the presence of the living room in almost all floor plans, our designed learning network begins with positioning a living room and continues by iteratively generating other rooms. Then, walls are first determined by an encoder-decoder network, and then they are refined to vector representations using dedicated rules. To effectively train our networks, we construct RPLAN - a manually collected large-scale densely annotated dataset of floor plans from real residential buildings. Intensive experiments, including formative user studies and comparisons, are conducted to illustrate the feasibility and efficacy of our proposed approach. By comparing the plausibility of different floor plans, we have observed that our method substantially outperforms existing methods, and in many cases our floor plans are comparable to human-created ones.

Considering the fact that points of interest on 3D shapes can be discriminated from a geometric perspective, it is reasonable to map the geometric signature of a point $p$p to a probability value encoding to what degree $p$p is a point of interest, especially for a specific class of 3D shapes. Based on the observation, we propose a three-phase algorithm for learning and predicting points of interest on 3D shapes by using multiple feature descriptors. Our algorithm requires two separate deep neural networks (stacked auto-encoders) to accomplish the task. During the first phase, we predict the membership of the given 3D shape according to a set of geometric descriptors using a deep neural network. After that, we train the other deep neural network to predict a probability distribution defined on the surface representing the possibility of a point being a point of interest. Finally, we use a manifold clustering technique to extract a set of points of interest as the output. Experimental results show superior detection performance of the proposed method over the previous state-of-the-art approaches.

Mesh denoising is a classical, yet not well-solved problem in digital geometry processing. The challenge arises from noise removal with the minimal disturbance of surface intrinsic properties (e.g., sharp features and shallow details). We propose a new patch normal co-filter (PcFilter) for mesh denoising. It is inspired by the geometry statistics which show that surface patches with similar intrinsic properties exist on the underlying surface of a noisy mesh. We model the PcFilter as a low-rank matrix recovery problem of similar-patch collaboration, aiming at removing different levels of noise, yet preserving various surface features. We generalize our model to pursue the low-rank matrix recovery in the kernel space for handling the nonlinear structure contained in the data. By making use of the block coordinate descent minimization and the specifics of a proximal based coordinate descent method, we optimize the nonlinear and nonconvex objective function efficiently. The detailed quantitative and qualitative results on synthetic and real data show that the PcFilter competes favorably with the state-of-the-art methods in surface accuracy and noise-robustness.

Compatible remeshing provides meshes with common connectivity structures. The existing compatible remeshing methods usually suffer from high computational cost or poor quality. In this paper, we present a fast method for computing compatible meshes with high quality. Given two closed, oriented, and topologically equivalent surfaces and a sparse set of corresponding landmarks, we first compute a bijective inter-surface mapping, from which compatible meshes are generated. We then improve the remeshing quality by using a volume-enhanced optimization. In contrast to previous work, our method designs a fast volume-enhanced improvement procedure that directly reduces the isometric distortion of the map between the compatible meshes. Our method also tries to preserve the shapes of the input meshes by projecting the vertices of the compatible meshes onto the input surfaces. Central to this approach is the use of the monotone preconditioned conjugate gradient method, which minimizes the energies effectively and efficiently. Compared with state-of-the-art methods, our method performs about one order of magnitude faster with better remeshing quality. We demonstrate the efficiency and efficacy of our method using various model pairs.

The normals of feature points, i.e., the intersection points of multiple smooth surfaces, are ambiguous and undefined. This paper presents a unified definition for point cloud normals of feature and non-feature points, which allows feature points to possess multiple normals. This definition facilitates several succeeding operations, such as feature points extraction and point cloud filtering. We also develop a feature preserving normal estimation method which outputs multiple normals per feature point. The core of the method is a pair consistency voting scheme. All neighbor point pairs vote for the local tangent plane. Each vote takes the fitting residuals of the pair of points and their preliminary normal consistency into consideration. Thus the pairs from the same subspace and relatively far off features dominate the voting. An adaptive strategy is designed to overcome sampling anisotropy. In addition, we introduce an error measure compatible with traditional normal estimators, and present the first benchmark for normal estimation, composed of 152 synthesized data with various features and sampling densities, and 288 real scans with different noise levels. Comprehensive and quantitative experiments show that our method generates faithful feature preserving normals and outperforms previous cutting edge normal estimation methods, including the latest deep learning based method.

The joint bilateral filter, which enables feature-preserving signal smoothing according to the structural information from a guidance, has been applied for various tasks in geometry processing. Existing methods either rely on a static guidance that may be inconsistent with the input and lead to unsatisfactory results, or a dynamic guidance that is automatically updated but sensitive to noises and outliers. Inspired by recent advances in image filtering, we propose a new geometry filtering technique called static/dynamic filter, which utilizes both static and dynamic guidances to achieve state-of-the-art results. The proposed filter is based on a nonlinear optimization that enforces smoothness of the signal while preserving variations that correspond to features of certain scales. We develop an efficient iterative solver for the problem, which unifies existing filters that are based on static or dynamic guidances. The filter can be applied to mesh face normals followed by vertex position update, to achieve scale-aware and feature-preserving filtering of mesh geometry. It also works well for other types of signals defined on mesh surfaces, such as texture colors. Extensive experimental results demonstrate the effectiveness of the proposed filter for various geometry processing applications such as mesh denoising, geometry feature enhancement, and texture color filtering.

3D printing, also called additive manufacturing, has been increasingly popular and printing efficiency has become more critical. To print artifacts faster with less material, thus leading to lighter and cheaper printed products, various types of void structureshave been designed and engineered inside of shape models. In this paper, we present a novel method for generating support-free elliptic hollowing for 3D shapes which can entirely avoid additional supporting structures. To achieve this, we perform the ellipse hollowing in one of the cross sectional polygons and then extrude the hollowed ellipses to the other parallel cross sections. To efficiently pack the ellipses in the polygon, we construct the Voronoi diagram of ellipses to reason the free-space around the ellipses and other geometric features by taking advantage of the available algorithm for the efficient and robust construction of the Voronoi diagram of circles. We demonstrate the effectiveness and feasibility of our proposed method by designing and printing support-free hollow for various 3D shapes using Poretron, the program which computes the hollow by embedding appropriate APIs of the Voronoi Diagram Machine library that is freely available from Voronoi Diagram Research Center. It takes a 3D mesh model and produces an STL file which can be either fed into a 3D printer or postprocessed.

Abstract We propose a hierarchical framework for the generation of building interiors. Our solution is based on a mixed integer quadratic programming (MIQP) formulation. We parametrize a layout by polygons that are further decomposed into small rectangles. We identify important high‐level constraints, such as room size, room position, room adjacency, and the outline of the building, and formulate them in a way that is compatible with MIQP and the problem parametrization. We also propose a hierarchical framework to improve the scalability of the approach. We demonstrate that our algorithm can be used for residential building layouts and can be scaled up to large layouts such as office buildings, shopping malls, and supermarkets. We show that our method is faster by multiple orders of magnitude than previous methods.

Abstract Mesh denoising is a fundamental problem in geometry processing. The main challenge is to preserve sharp features (such as edges and corners) and smooth regions (such as smoothly curved regions and fine details) while removing the noise. State‐of‐the‐art denoising methods still struggle with this issue. In this paper, we first propose a new variational model combining total variation and anisotropic Laplacian regularization to filter the normal vector field of the mesh. This model can preserve sharp features and simultaneously handle smooth regions well. Then, a new vertex updating scheme is presented to reconstruct the mesh according to the filtered face normals. It prevents the orientation ambiguity problem introduced by existing schemes. Experiments show that our denoising method outperforms all compared methods visually and quantitatively, especially for meshes consisting of both sharp features and smooth regions.

3D face reconstruction from a single image is a classical and challenging problem, with wide applications in many areas. Inspired by recent works in face animation from RGBD or monocular video inputs, we develop a novel method for reconstructing 3D faces from unconstrained 2D images, using a coarse-to-fine optimization strategy. First, a smooth coarse 3D face is generated from an example-based bilinear face model, by aligning the projection of 3D face landmarks with 2D landmarks detected from the input image. Afterwards, using local corrective deformation fields, the coarse 3D face is refined using photometric consistency constraints, resulting in a medium face shape. Finally, a shape-from-shading method is applied on the medium face to recover fine geometric details. Our method outperforms stateof- the-art approaches in terms of accuracy and detail recovery, which is demonstrated in extensive experiments using real world models and publicly available datasets.

To carry out autonomous 3D scanning and online reconstruction of unknown indoor scenes, one has to find a balance between global exploration of the entire scene and local scanning of the objects within it. In this work, we propose a novel approach, which provides object-aware guidance for autoscanning, for exploring, reconstructing, and understanding an unknown scene within one navigation pass. Our approach interleaves between object analysis to identify the next best object (NBO) for global exploration, and object-aware information gain analysis to plan the next best view (NBV) for local scanning. First, an objectness-based segmentation method is introduced to extract semantic objects from the current scene surface via a multi-class graph cuts minimization. Then, an object of interest (OOI) is identified as the NBO which the robot aims to visit and scan. The robot then conducts fine scanning on the OOI with views determined by the NBV strategy. When the OOI is recognized as a full object, it can be replaced by its most similar 3D model in a shape database. The algorithm iterates until all of the objects are recognized and reconstructed in the scene. Various experiments and comparisons have shown the feasibility of our proposed approach.

We propose a novel approach, calledProgressive Parameterizations, to compute foldover-free parameterizations with low isometric distortion on disk topology meshes. Instead of using the input mesh as a reference to define the objective function, we introduce a progressive reference that contains bounded distortion to the parameterized mesh and is as close as possible to the input mesh. After optimizing the bounded distortion energy between the progressive reference and the parameterized mesh, the parameterized mesh easily approaches the progressive reference, thereby also coming close to the input. By iteratively generating the progressive reference and optimizing the bounded distortion energy to update the parameterized mesh, our algorithm achieves high-quality parameterizations with strong practical reliability and high efficiency. We have demonstrated that our algorithm succeeds on a massive test data set containing over 20712 complex disk topology meshes. Compared to the state-of-the-art methods, our method has achieved higher computational efficiency and practical reliability.

Many computer graphics problems require computing geometric shapes subject to certain constraints. This often results in non-linear and non-convex optimization problems with globally coupled variables, which pose great challenge for interactive applications. Local-global solvers developed in recent years can quickly compute an approximate solution to such problems, making them an attractive choice for applications that prioritize efficiency over accuracy. However, these solvers suffer from lower convergence rate, and may take a long time to compute an accurate result. In this paper, we propose a simple and effective technique to accelerate the convergence of such solvers. By treating each local-global step as a fixed-point iteration, we apply Anderson acceleration, a well-established technique for fixed-point solvers, to speed up the convergence of a local-global solver. To address the stability issue of classical Anderson acceleration, we propose a simple strategy to guarantee the decrease of target energy and ensure its global convergence. In addition, we analyze the connection between Anderson acceleration and quasi-Newton methods, and show that the canonical choice of its mixing parameter is suitable for accelerating local-global solvers. Moreover, our technique is effective beyond classical local-global solvers, and can be applied to iterative methods with a common structure. We evaluate the performance of our technique on a variety of geometry optimization and physics simulation problems. Our approach significantly reduces the number of iterations required to compute an accurate result, with only a slight increase of computational cost per iteration. Its simplicity and effectiveness makes it a promising tool for accelerating existing algorithms as well as designing efficient new algorithms.

Computing spherical parameterizations for genus-zero closed surfaces is a fundamental task for geometric processing and computer graphics. Existing methods usually suffer from a lack of practical robustness or poor quality. In this paper, we present a practically robust method to compute high-quality spherical parameterizations with bijection and low isometric distortion. Our method is based on the hierarchical scheme containing mesh decimation and parameterization refinement. The practical robustness of our method relies on two novel techniques. The first one is a flat-to-extrusive decimation strategy, which contains two decimation error metrics to alleviate the difficulty of further mesh refinement. The second is a flexible group refinement technique that consists of flexible vertex insertion and efficient volumetric distortion minimization to control the maximum distortion. We convert the task of volumetric distortion minimization to one of tetrahedral mesh improvement to make the vertices distribute uniformly for efficient refinement. Compared with state-of-the-art methods, our method is more practically robust and possesses better mapping qualities. We demonstrate the efficacy of our method in spherical parameterization computations on a data set containing over five thousand complex models.

Offsetting-based hollowing is a solid modeling operation widely used in 3D printing, which can change the model's physical properties and reduce the weight by generating voids inside a model. However, a hollowing operation can lead to additional supporting structures for fabrication in interior voids, which cannot be removed. As a consequence, the result of a hollowing operation is affected by these additional supporting structures when applying the operation to optimize physical properties of different models. This paper proposes a support-free hollowing framework to overcome the difficulty of fabricating voids inside a solid. The challenge of computing a support-free hollowing is decomposed into a sequence of shape optimization steps, which are repeatedly applied to interior mesh surfaces. The optimization of physical properties in different applications can be easily integrated into our framework. Comparing to prior approaches that can generate support-free inner structures, our hollowing operation can reduce more volume of material and thus provide a larger solution space for physical optimization. Experimental tests are taken on a number of 3D models to demonstrate the effectiveness of this framework.

The most challenging problem in mesh denoising is to distinguish features from noise. Based on the robust guided normal estimation and alternate vertex updating strategy, we investigate a new feature-preserving mesh denoising method. To accurately capture local structures around features, we propose a corner-aware neighborhood (CAN) scheme. By combining both overall normal distribution of all faces in a CAN and individual normal influence of the interested face, we give a new consistency measuring method, which greatly improves the reliability of the estimated guided normals. As the noise level lowers, we take as guidance the previous filtered normals, which coincides with the emerging rolling guidance idea. In the vertex updating process, we classify vertices according to filtered normals at each iteration and reposition vertices of distinct types alternately with individual regularization constraints. Experiments on a variety of synthetic and real data indicate that our method adapts to various noise, both Gaussian and impulsive, no matter in the normal direction or in a random direction, with few triangles flipped.

We present a fully automatic method for finding geometrically consistent correspondences while discarding outliers from the candidate point matches in two images. Given a set of candidate matches provided by scale-invariant feature transform (SIFT) descriptors, which may contain many outliers, our goal is to select a subset of these matches retaining much more geometric information constructed by a mapping searched in the space of all diffeomorphisms. This problem can be formulated as a constrained optimization involving both the Beltrami coefficient (BC) term and quasi-conformal map, and solved by an efficient iterative algorithm based on the variable splitting method. In each iteration, we solve two subproblems, namely a linear system and linearly constrained convex quadratic programming. Our algorithm is simple and robust to outliers. We show that our algorithm enables producing more correct correspondences experimentally compared with state-of-the-art approaches.

Virtual reality applications prefer real walking to provide highly immersive presence than other locomotive methods. Mapping-based techniques are very effective for supporting real walking in small physical workspaces while exploring large virtual scenes. However, the existing methods for computing real walking maps suffer from poor quality due to distortion. In this paper, we present a novel divide-and-conquer method, called Smooth Assembly Mapping (SAM), to compute real walking maps with low isometric distortion for large-scale virtual scenes. First, the input virtual scene is decomposed into a set of smaller local patches. Then, a group of local patches is mapped together into a real workspace by minimizing a low isometric distortion energy with smoothness constraints between the adjacent patches. All local patches are mapped and assembled one by one to obtain a complete map. Finally, a global optimization is adopted to further reduce the distortion throughout the entire map. Our method easily handles teleportation technique by computing maps of individual regions and assembling them with teleporter conformity constraints. A large number of experiments, including formative user studies and comparisons, have shown that our method succeeds in generating high-quality real walking maps from large-scale virtual scenes to small real workspaces and is demonstrably superior to state-of-the-art methods.

Reconfigurable assemblies consist of a common set of parts that can be assembled into different forms for use in different situations. Designing these assemblies is a complex problem, since it requires a compatible decomposition of shapes with correspondence across forms, and a planning of well-matched joints to connect parts in each form. This paper presents computational methods as tools to assist the design and construction of reconfigurable assemblies, typically for furniture. There are three key contributions in this work. First, we present the compatible decomposition as a weakly-constrained dissection problem, and derive its solution based on a dynamic bipartite graph to construct parts across multiple forms; particularly, we optimize the parts reuse and preserve the geometric semantics. Second, we develop a joint connection graph to model the solution space of reconfigurable assemblies with part and joint compatibility across different forms. Third, we formulate the backward interlocking and multi-key interlocking models, with which we iteratively plan the joints consistently over multiple forms. We show the applicability of our approach by constructing reconfigurable furniture of various complexities, extend it with recursive connections to generate extensible and hierarchical structures, and fabricate a number of results using 3D printing, 2D laser cutting, and woodworking.

Wind-up toys are mechanical assemblies that perform intriguing motions driven by a simple spring motor. Due to the limited motor force and small body size, wind-up toys often employ higher pair joints of less frictional contacts and connector parts of nontrivial shapes to transfer motions. These unique characteristics make them hard to design and fabricate as compared to other automata. This paper presents a computational system to aid the design of wind-up toys, focusing on constructing a compact internal wind-up mechanism to realize user-requested part motions. Our key contributions include an analytical modeling of a wide variety of elemental mechanisms found in common wind-up toys, including their geometry and kinematics, conceptual design of wind-up mechanisms by computing motion transfer trees to realize the requested part motions, automatic construction of wind-up mechanisms by connecting multiple elemental mechanisms, and an optimization on the part and joint geometry with an objective of compacting the mechanism, reducing its weight, and avoiding collision. We use our system to design wind-up toys of various forms, fabricate a number of them using 3D printing, and show the functionality of various results.

Frame shapes, which are made of struts, have been widely used in many fields, such as art, sculpture, architecture, and geometric modeling, etc. An interest in robotic fabrication of frame shapes via spatial thermoplastic extrusion has been increasingly growing in recent years. In this paper, we present a novel algorithm to generate a feasible fabrication sequence for general frame shapes. To solve this non-trivial combinatorial problem, we develop a divide-and-conquer strategy that first decomposes the input frame shape into stable layers via a constrained sparse optimization model. Then we search a feasible sequence for each layer via a local optimization method together with a backtracking strategy. The generated sequence guarantees that the already-printed part is in a stable equilibrium state at all stages of fabrication, and that the 3D printing extrusion head does not collide with the printed part during the fabrication. Our algorithm has been validated by a built prototype robotic fabrication system made by a 6-axis KUKA robotic arm with a customized extrusion head. Experimental results demonstrate the feasibility and applicability of our algorithm.

This paper presents CofiFab, a coarse-to-fine 3D fabrication solution, combining 3D printing and 2D laser cutting for cost-effective fabrication of large objects at lower cost and higher speed. Our key approach is to first build coarse internal base structures within the given 3D object using laser cutting, and then attach thin 3D-printed parts, as an external shell, onto the base to recover the fine surface details. CofiFab achieves this with three novel algorithmic components. First, we formulate an optimization model to compute fabricatable polyhedrons of maximized volume, as the geometry of the internal base. Second, we devise a new interlocking scheme to tightly connect the laser-cut parts into a strong internal base, by iteratively building a network of nonorthogonal joints and interlocking parts around polyhedral corners. Lastly, we optimize the partitioning of the external object shell into 3D-printable parts, while saving support material and avoiding overhangs. Besides cost saving, these components also consider aesthetics, stability and balancing. Hence, CofiFab can efficiently produce large objects by assembly. To evaluate CofiFab, we fabricate objects of varying shapes and sizes, and show that CofiFab can significantly outperform previous methods.

Manifold is an important technique to model geometric objects with arbitrary topology. In this article, we propose a novel approach for constructing manifolds from discrete meshes based on sparse optimization. The local geometry for each chart is sparsely represented by a set of redundant atom functions, which have the flexibility to represent various geometries with varying smoothness. A global optimization is then proposed to guarantee compatible sparse representations in the overlapping regions of different charts. Our method can construct manifolds of varying smoothness including sharp features (creases, darts, or cusps). As an application, we can easily construct a skinning manifold surface from a given curve network. Examples show that our approach has much flexibility to generate manifold surfaces with good quality.

Scissor structure is used to generate deployable objects for space-saving in a variety of applications, from architecture to aerospace science. While deployment from a small, regular shape to a larger one is easy to design, we focus on a more challenging task: designing a planar scissor structure that deploys from a given source shape into a specific target shape. We propose a two-step constructive method to generate a scissor structure from a high-dimensional parameter space. Topology construction of the scissor structure is first performed to approximate the two given shapes, as well as to guarantee the deployment. Then the geometry of the scissor structure is optimized in order to minimize the connection deflections and maximize the shape approximation. With the optimized parameters, the deployment can be simulated by controlling an anchor scissor unit. Physical deployable objects are fabricated according to the designed scissor structures by using 3D printing or manual assembly. We show a number of results for different shapes to demonstrate that even with fabrication errors, our designed structures can deform fluently between the source and target shapes.

Abstract We present an adaptive slicing scheme for reducing the manufacturing time for 3D printing systems. Based on a new saliency‐based metric, our method optimizes the thicknesses of slicing layers to save printing time and preserve the visual quality of the printing results. We formulate the problem as a constrained ℓ0 optimization and compute the slicing result via a two‐step optimization scheme. To further reduce printing time, we develop a saliency‐based segmentation scheme to partition an object into subparts and then optimize the slicing of each subpart separately. We validate our method with a large set of 3D shapes ranging from CAD models to scanned objects. Results show that our method saves printing time by 30–40% and generates 3D objects that are visually similar to the ones printed with the finest resolution possible.

Feature learning for 3D shapes is challenging due to the lack of natural paramterization for 3D surface models. We adopt the multi-view depth image representation and propose Multi-View Deep Extreme Learning Machine (MVD-ELM) to achieve fast and quality projective feature learning for 3D shapes. In contrast to existing multi-view learning approaches, our method ensures the feature maps learned for different views are mutually dependent via shared weights and in each layer, their unprojections together form a valid 3D reconstruction of the input 3D shape through using normalized convolution kernels. These lead to a more accurate 3D feature learning as shown by the encouraging results in several applications. Moreover, the 3D reconstruction property enables clear visualization of the learned features, which further demonstrates the meaningfulness of our feature learning.

The joint bilateral filter is a variant of the standard bilateral filter, where the range kernel is evaluated using a guidance signal instead of the original signal. It has been successfully applied to various image processing problems, where it provides more flexibility than the standard bilateral filter to achieve high quality results. On the other hand, its success is heavily dependent on the guidance signal, which should ideally provide a robust estimation about the features of the output signal. Such a guidance signal is not always easy to construct. In this paper, we propose a novel mesh normal filtering framework based on the joint bilateral filter, with applications in mesh denoising. Our framework is designed as a two-stage process: first, we apply joint bilateral filtering to the face normals, using a properly constructed normal field as the guidance; afterwards, the vertex positions are updated according to the filtered face normals. We compute the guidance normal on a face using a neighboring patch with the most consistent normal orientations, which provides a reliable estimation of the true normal even with a high-level of noise. The effectiveness of our approach is validated by extensive experimental results.

We introduce a contextual descriptor which aims to provide a geometric description of the functionality of a 3D object in the context of a given scene. Differently from previous works, we do not regard functionality as an abstract label or represent it implicitly through an agent. Our descriptor, called interaction context or ICON for short, explicitly represents the geometry of object-to-object interactions. Our approach to object functionality analysis is based on the key premise that functionality should mainly be derived from interactions between objects and not objects in isolation. Specifically, ICON collects geometric and structural features to encode interactions between a central object in a 3D scene and its surrounding objects. These interactions are then grouped based on feature similarity, leading to a hierarchical structure. By focusing on interactions and their organization, ICON is insensitive to the numbers of objects that appear in a scene, the specific disposition of objects around the central object, or the objects' fine-grained geometry. With a series of experiments, we demonstrate the potential of ICON in functionality-oriented shape processing, including shape retrieval (either directly or by complementing existing shape descriptors), segmentation, and synthesis.

Most mesh denoising techniques utilize only either the facet normal field or the vertex normal field of a mesh surface. The two normal fields, though contain some redundant geometry information of the same model, can provide additional information that the other field lacks. Thus, considering only one normal field is likely to overlook some geometric features. In this paper, we take advantage of the piecewise consistent property of the two normal fields and propose an effective framework in which they are filtered and integrated using a novel method to guide the denoising process. Our key observation is that, decomposing the inconsistent field at challenging regions into multiple piecewise consistent fields makes the two fields complementary to each other and produces better results. Our approach consists of three steps: vertex classification, bi-normal filtering, and vertex position update. The classification step allows us to filter the two fields on a piecewise smooth surface rather than a surface that is smooth everywhere. Based on the piecewise consistence of the two normal fields, we filtered them using a piecewise smooth region clustering strategy. To benefit from the bi-normal filtering, we design a quadratic optimization algorithm for vertex position update. Experimental results on synthetic and real data show that our algorithm achieves higher quality results than current approaches on surfaces with multifarious geometric features and irregular surface sampling.

Computing geodesic distances on triangle meshes is a fundamental problem in computational geometry and computer graphics. To date, two notable classes of algorithms, the Mitchell-Mount-Papadimitriou (MMP) algorithm and the Chen-Han (CH) algorithm, have been proposed. Although these algorithms can compute exact geodesic distances if numerical computation is exact, they are computationally expensive, which diminishes their usefulness for large-scale models and/or time-critical applications. In this paper, we propose the fast wavefront propagation (FWP) framework for improving the performance of both the MMP and CH algorithms. Unlike the original algorithms that propagate only a single window (a data structure locally encodes geodesic information) at each iteration, our method organizes windows with a bucket data structure so that it can process a large number of windows simultaneously without compromising wavefront quality. Thanks to its macro nature, the FWP method is less sensitive to mesh triangulation than the MMP and CH algorithms. We evaluate our FWP-based MMP and CH algorithms on a wide range of large-scale real-world models. Computational results show that our method can improve the speed by a factor of 3-10.

Abstract We propose a fast method for 3D shape segmentation and labeling via Extreme Learning Machine (ELM). Given a set of example shapes with labeled segmentation, we train an ELM classifier and use it to produce initial segmentation for test shapes. Based on the initial segmentation, we compute the final smooth segmentation through a graph‐cut optimization constrained by the super‐face boundaries obtained by over‐segmentation and the active contours computed from ELM segmentation. Experimental results show that our method achieves comparable results against the state‐of‐the‐arts, but reduces the training time by approximately two orders of magnitude, both for face‐level and super‐face‐level, making it scale well for large datasets. Based on such notable improvement, we demonstrate the application of our method for fast online sequential learning for 3D shape segmentation at face level, as well as realtime sequential learning at super‐face level.

A novel representation of a triangular mesh surface using a set of scale-invariant measures is proposed. The measures consist of angles of the triangles (triangle angles) and dihedral angles along the edges (edge angles) which are scale and rigidity independent. The vertex coordinates for a mesh give its scale-invariant measures, unique up to scale, rotation, and translation. Based on the representation of mesh using scale-invariant measures, a two-step iterative deformation algorithm is proposed, which can arbitrarily edit the mesh through simple handles interaction. The algorithm can explicitly preserve the local geometric details as much as possible in different scales even under severe editing operations including rotation, scaling, and shearing. The efficiency and robustness of the proposed algorithm are demonstrated by examples.

Many geometry processing applications are sensitive to noise and sharp features. Although there are a number of works on detecting noise and sharp features in the literature, they are heuristic. On one hand, traditional denoising methods use filtering operators to remove noise, however, they may blur sharp features and shrink the object. On the other hand, noise makes detection of features, which relies on computation of differential properties, unreliable and unstable. Therefore, detecting noise and features on discrete surfaces still remains challenging. In this article, we present an approach for decoupling noise and features on 3D shapes. Our approach consists of two phases. In the first phase, a base mesh is estimated from the input noisy data by a global Laplacian regularization denoising scheme. The estimated base mesh is guaranteed to asymptotically converge to the true underlying surface with probability one as the sample size goes to infinity. In the second phase, an ℓ 1 -analysis compressed sensing optimization is proposed to recover sharp features from the residual between base mesh and input mesh. This is based on our discovery that sharp features can be sparsely represented in some coherent dictionary which is constructed by the pseudo-inverse matrix of the Laplacian of the shape. The features are recovered from the residual in a progressive way. Theoretical analysis and experimental results show that our approach can reliably and robustly remove noise and extract sharp features on 3D shapes.

We present a method to complete missing structures in facade layouts. Starting from an abstraction of the partially observed layout as a set of shapes, we can propose one or multiple possible completed layouts. Structure completion with large missing parts is an ill-posed problem. Therefore, we combine two sources of information to derive our solution: the observed shapes and a database of complete layouts. The problem is also very difficult, because shape positions and attributes have to be estimated jointly. Our proposed solution is to break the problem into two components: a statistical model to evaluate layouts and a planning algorithm to generate candidate layouts. This ensures that the completed result is consistent with the observation and the layouts in the database.

Surface reconstruction from point cloud is of great practical importance in computer graphics. Existing methods often realize reconstruction via a few phases with respective goals, whose integration may not give an optimal solution. In this paper, to avoid the inherent limitations of multi-phase processing in the prior art, we propose a unified framework that treats geometry and connectivity construction as one joint optimization problem. The framework is based on dictionary learning in which the dictionary consists of the vertices of the reconstructed triangular mesh and the sparse coding matrix encodes the connectivity of the mesh. The dictionary learning is formulated as a constrained ℓ 2,q -optimization (0 < q < 1), aiming to find the vertex position and triangulation that minimize an energy function composed of point-to-mesh metric and regularization. Our formulation takes many factors into account within the same framework, including distance metric, noise/outlier resilience, sharp feature preservation, no need to estimate normal, etc., thus providing a global and robust algorithm that is able to efficiently recover a piecewise smooth surface from dense data points with imperfections. Extensive experiments using synthetic models, real world models, and publicly available benchmark show that our method outperforms the state-of-the-art in terms of accuracy, robustness to noise and outliers, geometric feature and detail preservation, and mesh connectivity.

Barycentric coordinates yield a powerful and yet simple paradigm to interpolate data values on polyhedral domains. They represent interior points of the domain as an affine combination of a set of control points, defining an interpolation scheme for any function defined on a set of control points. Numerous barycentric coordinate schemes have been proposed satisfying a large variety of properties. However, they typically define interpolation as a combination ofallcontrol points. Thus alocalchange in the value at a single control point will create aglobalchange by propagation into the whole domain. In this context, we present a family oflocal barycentric coordinates(LBC), which select for each interior point a small set of control points and satisfy common requirements on barycentric coordinates, such as linearity, non-negativity, and smoothness. LBC are achieved through a convex optimization based on total variation, and provide a compact representation that reduces memory footprint and allows for fast deformations. Our experiments show that LBC provide more local and finer control on shape deformation than previous approaches, and lead to more intuitive deformation results.

Abstract Modeling 3D objects is difficult, especially for the user who lacks the knowledge on 3D geometry or even on 2D sketching. In this paper, we present a novel sketch‐based modeling system which allows novice users to create 3D custom models by assembling parts based on a database of pre‐segmented 3D models. Different from previous systems, our system supports the user with visualized and meaningfulshadow guidanceunder his strokes dynamically to guide the user to convey his design concept easily and quickly. Our system interprets the user's strokes as similarity queries into database to generate the shadow image for guiding the user's further drawing and returns the 3D candidate parts for modeling simultaneously. Moreover, our system preserves the high‐level structure in generated models based on prior knowledge pre‐analyzed from the database, and allows the user to create custom parts with geometric variations. We demonstrate the applicability and effectiveness of our modeling system with human subjects and present various models designed using our system.

Abstract The emergence of laser/LiDAR sensors, reliable multi‐view stereo techniques and more recently consumer depth cameras have brought point clouds to the forefront as a data format useful for a number of applications. Unfortunately, the point data from those channels often incur imperfection, frequently contaminated with severe outliers and noise. This paper presents a robust consolidation algorithm for low‐quality point data from outdoor scenes, which essentially consists of two steps: 1) outliers filtering and 2) noise smoothing. We first design a connectivity‐based scheme to evaluate outlierness and thereby detect sparse outliers. Meanwhile, a clustering method is used to further remove small dense outliers. Both outlier removal methods are insensitive to the choice of the neighborhood size and the levels of outliers. Subsequently, we propose a novel approach to estimate normals for noisy points based on robust partial rankings, which is the basis of noise smoothing. Accordingly, a fast approach is exploited to smooth noise, while preserving sharp features. We evaluate the effectiveness of the proposed method on the point clouds from a variety of outdoor scenes.

3D printers have become popular in recent years and enable fabrication of custom objects for home users. However, the cost of the material used in printing remains high. In this paper, we present an automatic solution to design a skin-frame structure for the purpose of reducing the material cost in printing a given 3D object. The frame structure is designed by an optimization scheme which significantly reduces material volume and is guaranteed to be physically stable, geometrically approximate, and printable. Furthermore, the number of struts is minimized by solving an l 0 sparsity optimization. We formulate it as a multi-objective programming problem and an iterative extension of the preemptive algorithm is developed to find a compromise solution. We demonstrate the applicability and practicability of our solution by printing various objects using both powder-type and extrusion-type 3D printers. Our method is shown to be more cost-effective than previous works.

Abstract We present a novel algorithm for automatically co‐segmenting a set of shapes from a common family into consistent parts. Starting from over‐segmentations of shapes, our approach generates the segmentations by grouping the primitive patches of the shapes directly and obtains their correspondences simultaneously. The core of the algorithm is to compute an affinity matrix where each entry encodes the similarity between two patches, which is measured based on the geometric features of patches. Instead of concatenating the different features into one feature descriptor, we formulate co‐segmentation into a subspace clustering problem in multiple feature spaces. Specifically, to fuse multiple features, we propose a new formulation of optimization with a consistent penalty, which facilitates both the identification of most similar patches and selection of master features for two similar patches. Therefore the affinity matrices for various features are sparsity‐consistent and the similarity between a pair of patches may be determined by part of (instead of all) features. Experimental results have shown how our algorithm jointly extracts consistent parts across the collection in a good manner.

We present an algorithm for multi-scale partial intrinsic symmetry detection over 2D and 3D shapes, where the scale of a symmetric region is defined by intrinsic distances between symmetric points over the region. To identify prominent symmetric regions which overlap and vary in form and scale, we decouple scale extraction and symmetry extraction by performing two levels of clustering. First, significant symmetry scales are identified by clustering sample point pairs from an input shape. Since different point pairs can share a common point, shape regions covered by points in different scale clusters can overlap. We introduce the symmetry scale matrix (SSM), where each entry estimates the likelihood two point pairs belong to symmetries at the same scale. The pair-to-pair symmetry affinity is computed based on a pair signature which encodes scales. We perform spectral clustering using the SSM to obtain the scale clusters. Then for all points belonging to the same scale cluster, we perform the second-level spectral clustering, based on a novel point-to-point symmetry affinity measure, to extract partial symmetries at that scale. We demonstrate our algorithm on complex shapes possessing rich symmetries at multiple scales.

Blue noise point sampling is one of the core algorithms in computer graphics. In this paper, we present a new and versatile variational framework for generating point distributions with high-quality blue noise characteristics while precisely adapting to given density functions. Different from previous approaches based on discrete settings of capacity-constrained Voronoi tessellation, we cast the blue noise sampling generation as a variational problem with continuous settings. Based on an accurate evaluation of the gradient of an energy function, an efficient optimization is developed which delivers significantly faster performance than the previous optimization-based methods. Our framework can easily be extended to generating blue noise point samples on manifold surfaces and for multi-class sampling. The optimization formulation also allows us to naturally deal with dynamic domains, such as deformable surfaces, and to yield blue noise samplings with temporal coherence. We present experimental results to validate the efficacy of our variational framework. Finally, we show a variety of applications of the proposed methods, including nonphotorealistic image stippling, color stippling, and blue noise sampling on deformable surfaces.

Abstract We introduce symmetry hierarchy of man‐made objects, a high‐level structural representation of a 3D model providing a symmetry‐induced, hierarchical organization of the model's constituent parts. Given an input mesh, we segment it into primitive parts and build an initial graph which encodes inter‐part symmetries and connectivity relations, as well as self‐symmetries in individual parts. The symmetry hierarchy is constructed from the initial graph via recursive graph contraction which either groups parts by symmetry or assembles connected sets of parts. The order of graph contraction is dictated by a set of precedence rules designed primarily to respect the law of symmetry in perceptual grouping and the principle of compactness of representation. We show that symmetry hierarchy naturally implies a hierarchical segmentation that is more meaningful than those produced by local geometric considerations. We also develop an application of symmetry hierarchies for structural shape editing.

ABSTRACT We present a novel sketch‐based tool, called iCutter (short for intelligent cutter), for cutting out semantic parts of 3D shapes. When a user performs a cutting task, he only needs to draw a freehand stroke to roughly specify where cuts should be made without much attention. Then, iCutter intelligently returns the best cut that meets the user's intention and expectation. We develop a novel scheme for selecting the optimal isoline from a well‐designed scalar field induced from the input stroke, which respects the part saliency as well as the input stroke. We demonstrate various examples to illustrate the flexibility and applicability of our iCutter tool. Copyright © 2011 John Wiley & Sons, Ltd.

Revealing the sketching sequence of a line drawing can be visually intriguing and used for video-based storytelling. Typically this is enabled based on tedious recording of artists' drawing process. We demonstrate that it is often possible to estimate a reasonable drawing order from a static line drawing with clearly defined shape geometry, which looks plausible to a human viewer. We map the key principles of drawing order from drawing cognition to computational procedures in our framework. Our system produces plausible animated constructions of input line drawings, with no or little user intervention. We test our algorithm on a range of input sketches, with varying degree of complexity and structure, and evaluate the results via a user study. We also present applications to gesture drawing synthesis and drawing animation creation especially in the context of video scribing.

We introduce an algorithm for 3D object modeling where the user draws creative inspiration from an object captured in a single photograph. Our method leverages the rich source of photographs for creative 3D modeling. However, with only a photo as a guide, creating a 3D model from scratch is a daunting task. We support the modeling process by utilizing an available set of 3D candidate models. Specifically, the user creates a digital 3D model as a geometric variation from a 3D candidate. Our modeling technique consists of two major steps. The first step is a user-guided image-space object segmentation to reveal the structure of the photographed object. The core step is the second one, in which a 3D candidate is automatically deformed to fit the photographed target under the guidance of silhouette correspondence. The set of candidate models have been pre-analyzed to possess useful high-level structural information, which is heavily utilized in both steps to compensate for the ill-posedness of the analysis and modeling problems based only on content in a single image. Equally important, the structural information is preserved by the geometric variation so that the final product is coherent with its inherited structural information readily usable for subsequent model refinement or processing.

Abstract Aesthetic images evoke an emotional response that transcends mere visual appreciation. In this work we develop a novel computational means for evaluating the composition aesthetics of a given image based on measuring several well‐grounded composition guidelines. A compound operator of crop‐and‐retarget is employed to change the relative position of salient regions in the image and thus to modify the composition aesthetics of the image. We propose an optimization method for automatically producing a maximally‐aesthetic version of the input image. We validate the performance of the method and show its effectiveness in a variety of experiments.

Man-made objects often consist of detailed and interleaving structures, which are created using cane, coils, metal wires, rods, etc. The delicate structures, although manufactured using simple procedures, are challenging to scan and reconstruct. We observe that such structures are inherently 1D, and hence are naturally represented using an arrangement of generating curves. We refer to the resultant surfaces as arterial surfaces. In this paper we approach for analyzing, reconstructing, and manipulating such arterial surfaces. The core of the algorithm is a novel deformable model, called arterial snake , that simultaneously captures the topology and geometry of the arterial objects. The recovered snakes produce a natural decomposition of the raw scans, with the decomposed parts often capturing meaningful object sections. We demonstrate the robustness of our algorithm on a variety of arterial objects corrupted with noise, outliers, and with large parts missing. We present a range of applications including reconstruction, topology repairing, and manipulation of arterial surfaces by directly controlling the underlying curve network and the associated sectional profiles, which are otherwise challenging to perform.

We present an easy-to-use image retouching technique for realistic reshaping of human bodies in a single image. A model-based approach is taken by integrating a 3D whole-body morphable model into the reshaping process to achieve globally consistent editing effects. A novel body-aware image warping approach is introduced to reliably transfer the reshaping effects from the model to the image, even under moderate fitting errors. Thanks to the parametric nature of the model, our technique parameterizes the degree of reshaping by a small set of semantic attributes, such as weight and height. It allows easy creation of desired reshaping effects by changing the full-body attributes, while producing visually pleasing results even for loosely-dressed humans in casual photographs with a variety of poses and shapes.

We present a novel approach to localization of sensors in a network given a subset of noisy inter-sensor distances. The algorithm is based on “stitching” together local structures by solving an optimization problem requiring the structures to fit together in an “As-Rigid-As-Possible” manner, hence the name ARAP. The local structures consist of reference “patches” and reference triangles, both obtained from inter-sensor distances. We elaborate on the relationship between the ARAP algorithm and other state-of-the-art algorithms, and provide experimental results demonstrating that ARAP is significantly less sensitive to sparse connectivity and measurement noise. We also show how ARAP may be distributed.

The essence of a 3D shape can often be well captured by its salient feature curves. In this paper, we explore the use of salient curves in synthesizing intuitive, shape-revealing textures on surfaces. Our texture synthesis is guided by two principles: matching the direction of the texture patterns to those of the salient curves, and aligning the prominent feature lines in the texture to the salient curves exactly. We have observed that textures synthesized by these principles not only fit naturally to the surface geometry, but also visually reveal, even reinforce, the shape's essential characteristics. We call these feature-aligned shape texturing . Our technique is fully automatic, and introduces two novel technical components in vector-field-guided texture synthesis: an algorithm that orients the salient curves on a surface for constrained vector field generation, and a feature-to-feature texture optimization.

While many 3D objects exhibit various forms of global symmetries, prominent intrinsic symmetries which exist only on parts of an object are also well recognized. Such partial symmetries are often seen as more natural than a global one, even when the symmetric parts are under complex pose. We introduce an algorithm to extract partial intrinsic reflectional symmetries (PIRS) of a 3D shape. Given a closed 2-manifold mesh, we develop a voting scheme to obtain an intrinsic reflectional symmetry axis (IRSA) transform, which is a scalar field over the mesh that accentuates prominent IRSAs of the shape. We then extract a set of explicit IRSA curves on the shape based on a refined measure of local reflectional symmetry support along a curve. The iterative refinement procedure combines IRSA-induced region growing and region-constrained symmetry support refinement to improve accuracy and address potential issues arising from rotational symmetries in the shape. We show how the extracted IRSA curves can be incorporated into a conventional mesh segmentation scheme so that the implied symmetry cues can be utilized to obtain more meaningful results. We also demonstrate the use of IRSA curves for symmetry-driven part repair.

Abstract We present a novel approach to parameterize a mesh with disk topology to the plane in a shape‐preserving manner. Our key contribution is a local/global algorithm, which combines a local mapping of each 3D triangle to the plane, using transformations taken from a restricted set, with a global “stitch” operation of all triangles, involving a sparse linear system. The local transformations can be taken from a variety of families, e.g. similarities or rotations, generating different types of parameterizations. In the first case, the parameterization tries to force each 2D triangle to be an as‐similar‐as‐possible version of its 3D counterpart. This is shown to yield results identical to those of the LSCM algorithm. In the second case, the parameterization tries to force each 2D triangle to be an as‐rigid‐as‐possible version of its 3D counterpart. This approach preserves shape as much as possible. It is simple, effective, and fast, due to pre‐factoring of the linear system involved in the global phase. Experimental results show that our approach provides almost isometric parameterizations and obtains more shape‐preserving results than other state‐of‐the‐art approaches. We present also a more general “hybrid” parameterization model which provides a continuous spectrum of possibilities, controlled by a single parameter. The two cases described above lie at the two ends of the spectrum. We generalize our local/global algorithm to compute these parameterizations. The local phase may also be accelerated by parallelizing the independent computations per triangle.

This paper presents a novel parameterization method for a non-closed triangular mesh. For every flattened 1-ring neighbors, we choose a local coordinate frame, and the local geometry structure is represented as local parametric coordinates. Then the global optimal parametric coordinates are attained by aligning all the local parametric planes while preserving the local structure as much as possible. The boundary conditions are not necessary in our method, thus no high distortion appears around the boundary, and distortion is uniformly distributed over parametric domain. In addition, our method can operate directly on mesh surface which has holes without any preprocessing of surface partition. Furthermore, linear constraints are allowed in the parameterization in a least squares sense.

Abstract We present Easy Mesh Cutting, an intuitive and easy‐to‐use mesh cutout tool. Users can cut meaningful components from meshes by simply drawing freehand sketches on the mesh. Our system provides instant visual feedback to obtain the cutting results based on an improved region growing algorithm using a feature sensitive metric. The cutting boundary can be automatically optimized or easily edited by users. Extensive experimentation shows that our approach produces good cutting results while requiring little skill or effort from the user and provides a good user experience. Based on the easy mesh cutting framework, we introduce two applications including sketch‐based mesh editing and mesh merging for geometry processing. Categories and Subject Descriptors (according to ACM CCS): I.3.5 [Computer Graphics]: Geometric algorithms, languages, and systems

Recently, differential information as local intrinsic feature descriptors has been used for mesh editing. Given certain user input as constraints, a deformed mesh is reconstructed by minimizing the changes in the differential information. Since the differential information is encoded in a global coordinate system, it must somehow be transformed to fit the orientations of details in the deformed surface, otherwise distortion will appear. We observe that visually pleasing deformed meshes should preserve both local parameterization and geometry details. We propose to encode these two types of information in the dual mesh domain due to the simplicity of the neighborhood structure of dual mesh vertices. Both sets of information are nondirectional and nonlinearly dependent on the vertex positions. Thus, we present a novel editing framework that iteratively updates both the primal vertex positions and the dual Laplacian coordinates to progressively reduce distortion in parametrization and geometry. Unlike previous related work, our method can produce visually pleasing deformations with simple user interaction, requiring only the handle positions, not local frames at the handles.

This paper presents a novel approach for surface smoothing with feature preservation on arbitrary meshes. Laplacian operator is performed in a global way over the mesh. The surface smoothing is formulated as a quadratic optimization problem, which is easily solved a sparse linear system. The cost function to be optimized penalizes deviations from the global Laplacian operator while maintaining the overall shape of the original mesh. The features of the original mesh can be preserved by adding feature constraints and barycenter constraints in the system. Our approach is simple, non-iterative, fast, and does not cause surface shrinkage and distortion. Many experimental results are presented to show the applicability and flexibility of the approach.

This paper presents a novel approach for constructing a piecewise triangular cubic polynomial surface with C/sup 1/ continuity around a common corner vertex. A C/sup 1/ continuity condition between two cubic triangular patches is first derived using mixed directional derivatives. An approach for constructing a surface with C/sup 1/ continuity around a corner is then developed. Our approach is easy and fast with the virtue of cubic reproduction, local shape controllability, C/sup 2/ continuous at the corner vertex. Some experimental results are presented to show the applicability and flexibility of the approach.

This paper presents an approach for establishing vertex correspondences between two planar shapes. Correspondences are established between the perceptual feature points extracted from both source and target shapes. A similarity metric between two feature points is defined using the intrinsic properties of their local neighborhoods. The optimal correspondence is found by an efficient dynamic programming technique. Our approach treats shape noise by allowing discarding small feature points, which introduces skips in the traversal of the dynamic programming graph. Our method is fast, feature preserving, and invariant to geometric transformations. We demonstrate the superiority of our approach over other approaches by experimental results.

There exist many computer graphics techniques which could achieve high quality tree generation. However, only few works focus on realistic modeling of tree bark. Difficulties lie in the complex appearance of the bark surfaces from a single image. We address three main issues here: feature specification; height field assignment; and texture correction. For feature specification, we use texton channel analysis to specify a variant of common bark features, inluding ironbark, vertical and horizontal fractures, tessellation, furrowed cork, and lenticels. For height field assignment, we develop an intuitive and easy-to-use user interface (UI). Here similarity-based texture editing is used for assigning height fields within a texton channel mask. For texture correction, we use the modeled height fields to eliminate the underlying lighting effects in a captured texture. Our modeling system is image-based: it takes as input a bark image and produces as output a textured height field representing a bark sample. We demonstrate that out method is an effective and easy-to-use technique to interactively model a variety of photo realistic bark surfaces.

The bidirectional texture function (BTF) is a 6D function that can describe textures arising from both spatially-variant surface reflectance and surface mesostructures. In this paper, we present an algorithm for synthesizing the BTF on an arbitrary surface from a sample BTF. A main challenge in surface BTF synthesis is the requirement of a consistent mesostructure on the surface, and to achieve that we must handle the large amount of data in a BTF sample. Our algorithm performs BTF synthesis based on surface textons, which extract essential information from the sample BTF to facilitate the synthesis. We also describe a general search strategy, called the k-coherent search, for fast BTF synthesis using surface textons. A BTF synthesized using our algorithm not only looks similar to the BTF sample in all viewing/lighthing conditions but also exhibits a consistent mesostructure when viewing and lighting directions change. Moreover, the synthesized BTF fits the target surface naturally and seamlessly. We demonstrate the effectiveness of our algorithm with sample BTFs from various sources, including those measured from real-world textures.

This paper presents the algorithms for calculating the control points of Hermite polynomial approximation to rational Bezier curves and the recursive formulae between different degree Hermite polynomial approximations based on the close relationship between hybrid polynomial approximation and Hermite polynomial approximation. The error bound for the Hermite polynomial approximation is also obtained.