Relation contexts have been proved to be useful for many challenging vision tasks. In the field of 3D object detection, previous methods have been taking the advantage of context encoding, graph embedding, or explicit relation reasoning to extract relation contexts. However, there exist inevitably redundant relation contexts due to noisy or low-quality proposals. In fact, invalid relation contexts usually indicate underlying scene misunderstanding and ambiguity, which may, on the contrary, reduce the performance in complex scenes. Inspired by recent attention mechanism like Transformer, we propose a novel 3D attention-based relation module (ARM3D). It encompasses object-aware relation reasoning to extract pair-wise relation contexts among qualified proposals and an attention module to distribute attention weights towards different relation contexts. In this way, ARM3D can take full advantage of the useful relation contexts and filter those less relevant or even confusing contexts, which mitigates the ambiguity in detection. We have evaluated the effectiveness of ARM3D by plugging it into several state-of-the-art 3D object detectors and showing more accurate and robust detection results. Extensive experiments show the capability and generalization of ARM3D on 3D object detection. Our source code is available at https://github.com/lanlan96/ARM3D .

Feature learning for 3D shapes is challenging due to the lack of natural paramterization for 3D surface models. We adopt the multi-view depth image representation and propose Multi-View Deep Extreme Learning Machine (MVD-ELM) to achieve fast and quality projective feature learning for 3D shapes. In contrast to existing multi-view learning approaches, our method ensures the feature maps learned for different views are mutually dependent via shared weights and in each layer, their unprojections together form a valid 3D reconstruction of the input 3D shape through using normalized convolution kernels. These lead to a more accurate 3D feature learning as shown by the encouraging results in several applications. Moreover, the 3D reconstruction property enables clear visualization of the learned features, which further demonstrates the meaningfulness of our feature learning.

Abstract We propose a fast method for 3D shape segmentation and labeling via Extreme Learning Machine (ELM). Given a set of example shapes with labeled segmentation, we train an ELM classifier and use it to produce initial segmentation for test shapes. Based on the initial segmentation, we compute the final smooth segmentation through a graph‐cut optimization constrained by the super‐face boundaries obtained by over‐segmentation and the active contours computed from ELM segmentation. Experimental results show that our method achieves comparable results against the state‐of‐the‐arts, but reduces the training time by approximately two orders of magnitude, both for face‐level and super‐face‐level, making it scale well for large datasets. Based on such notable improvement, we demonstrate the application of our method for fast online sequential learning for 3D shape segmentation at face level, as well as realtime sequential learning at super‐face level.

ABSTRACT Arthroscopic surgeries, which are widely used for anterior cruciate ligament (ACL) reconstruction, not only require advanced hand–eye coordination but also involve complicated surgical procedure, necessitating simulation‐based training for surgeons. This paper describes a surgical procedure simulation system for the training of arthroscopic ACL reconstruction. Different from existing simulation‐based training systems for basic surgical skills, this system provides a complete simulation for the entire procedure of arthroscopic ACL reconstruction, involving operations such as puncturing, probing, incision, and drilling. In this system, we employ a linear elastic finite element method and position‐based dynamics for deformable modeling. Simplified vertex duplicating method and an implementation of real‐time Boolean operations are proposed for the topological change of tissue models involved in the incision simulation and tunnel construction. Two specially designed force feedback models are introduced for the haptic rendering of probing and drilling operations. By using these fast and stable simulation methods, this system is able to provide real‐time realistic graphical and haptic feedback to the user, making it efficient and practical for training purpose. Feedback from the surgeon trainees shows that this system is very effective in training not only for basic surgical skills but also for the complex surgical procedure. Copyright © 2012 John Wiley & Sons, Ltd.

Existing techniques for animation of object fracture have a basic assumption that the object materials are homogeneous while most materials in real world are heterogeneous. To solve this problem, we introduce a novel method called MCA to simulate the fracture of heterogeneous materials. The method of movable cellular automata (MCA) is based on the discrete concept and provides advantages both of classical cellular automaton and discrete element methods. First, the object is represented as discrete sphere particles, named movable cellular automata, and then the MCA method is used to simulate the material and physical properties as well as to determine when and where the fracture occurs. To meet the real-time requirement, the MCA simulation is performed on GPU with CUDA in order to accelerate the complex calculation of automata's physical properties. After simulation, the results are directly sent to the vertex buffer object (VBO) for rendering to avoid the costly communication between CPU and GPU. The experimental results show the feasibility and effectiveness of the proposed method.

Abstract We introduce symmetry hierarchy of man‐made objects, a high‐level structural representation of a 3D model providing a symmetry‐induced, hierarchical organization of the model's constituent parts. Given an input mesh, we segment it into primitive parts and build an initial graph which encodes inter‐part symmetries and connectivity relations, as well as self‐symmetries in individual parts. The symmetry hierarchy is constructed from the initial graph via recursive graph contraction which either groups parts by symmetry or assembles connected sets of parts. The order of graph contraction is dictated by a set of precedence rules designed primarily to respect the law of symmetry in perceptual grouping and the principle of compactness of representation. We show that symmetry hierarchy naturally implies a hierarchical segmentation that is more meaningful than those produced by local geometric considerations. We also develop an application of symmetry hierarchies for structural shape editing.

We introduce an algorithm for 3D object modeling where the user draws creative inspiration from an object captured in a single photograph. Our method leverages the rich source of photographs for creative 3D modeling. However, with only a photo as a guide, creating a 3D model from scratch is a daunting task. We support the modeling process by utilizing an available set of 3D candidate models. Specifically, the user creates a digital 3D model as a geometric variation from a 3D candidate. Our modeling technique consists of two major steps. The first step is a user-guided image-space object segmentation to reveal the structure of the photographed object. The core step is the second one, in which a 3D candidate is automatically deformed to fit the photographed target under the guidance of silhouette correspondence. The set of candidate models have been pre-analyzed to possess useful high-level structural information, which is heavily utilized in both steps to compensate for the ill-posedness of the analysis and modeling problems based only on content in a single image. Equally important, the structural information is preserved by the geometric variation so that the final product is coherent with its inherited structural information readily usable for subsequent model refinement or processing.

We perform co-analysis of a set of man-made 3D objects to allow the creation of novel instances derived from the set. We analyze the objects at the part level and treat the anisotropic part scales as a shape style. The co-analysis then allows style transfer to synthesize new objects. The key to co-analysis is part correspondence, where a major challenge is the handling of large style variations and diverse geometric content in the shape set. We propose style-content separation as a means to address this challenge. Specifically, we define a correspondence-free style signature for style clustering. We show that confining analysis to within a style cluster facilitates tasks such as co-segmentation, content classification, and deformation-driven part correspondence. With part correspondence between each pair of shapes in the set, style transfer can be easily performed. We demonstrate our analysis and synthesis results on several sets of man-made objects with style and content variations.

The essence of a 3D shape can often be well captured by its salient feature curves. In this paper, we explore the use of salient curves in synthesizing intuitive, shape-revealing textures on surfaces. Our texture synthesis is guided by two principles: matching the direction of the texture patterns to those of the salient curves, and aligning the prominent feature lines in the texture to the salient curves exactly. We have observed that textures synthesized by these principles not only fit naturally to the surface geometry, but also visually reveal, even reinforce, the shape's essential characteristics. We call these feature-aligned shape texturing . Our technique is fully automatic, and introduces two novel technical components in vector-field-guided texture synthesis: an algorithm that orients the salient curves on a surface for constrained vector field generation, and a feature-to-feature texture optimization.

While many 3D objects exhibit various forms of global symmetries, prominent intrinsic symmetries which exist only on parts of an object are also well recognized. Such partial symmetries are often seen as more natural than a global one, even when the symmetric parts are under complex pose. We introduce an algorithm to extract partial intrinsic reflectional symmetries (PIRS) of a 3D shape. Given a closed 2-manifold mesh, we develop a voting scheme to obtain an intrinsic reflectional symmetry axis (IRSA) transform, which is a scalar field over the mesh that accentuates prominent IRSAs of the shape. We then extract a set of explicit IRSA curves on the shape based on a refined measure of local reflectional symmetry support along a curve. The iterative refinement procedure combines IRSA-induced region growing and region-constrained symmetry support refinement to improve accuracy and address potential issues arising from rotational symmetries in the shape. We show how the extracted IRSA curves can be incorporated into a conventional mesh segmentation scheme so that the implied symmetry cues can be utilized to obtain more meaningful results. We also demonstrate the use of IRSA curves for symmetry-driven part repair.

Abstract In this paper, we propose a fast and stable method for 2D shape deformation based on rigid square matching. Our method utilizes uniform quadrangular control meshes for 2D shapes and tries to maintain the rigidity of each square in the control mesh during user manipulation. A rigid shape matching method is performed to find an optimal pure rotational transformation for each square in the control mesh. An iterative solver is proposed to compute the final deformation result for the entire control mesh by minimizing the difference between the deformed vertices and their counterparts in the neighboring rigid square. The deformation result on the 2D shape is as rigid as possible and the details of the shape are preserved well. As extensions, we present a shape‐aware splitting method to improve the deformation effect for coarse meshes and a simple sketch‐based clustering method for skeletal deformation. Experiments with various 2D shapes show that our method is efficient and easy to use, and can provide physically plausible result for shapes of objects in real world. Therefore, our shape deformation method is especially suitable for applications in cartoon character animation. Copyright © 2008 John Wiley & Sons, Ltd.