Automatic object viewpoint estimation from a single image is an important but challenging problem in machine intelligence community. Although impressive performance has been achieved, current state-of-the-art methods still have difficulty to deal with the visual ambiguity and structure ambiguity in real world images. To tackle these problems, a novel Appearance-and-Structure Fusion network, which we call it ASFnet that estimates viewpoint by fusing both appearance and structure information, is proposed in this paper. The structure information is encoded by precise semantic keypoints and can help address the visual ambiguity. Meanwhile, distinguishable appearance features contribute to overcoming the structure ambiguity. Our ASFnet integrates an appearance path and a structure path to an end-to-end network and allows deep features effectively share supervision from both the two complementary aspects. A convolutional layer is learned to fuse the two path results adaptively. To balance the influence from the two supervision sources, a piecewise loss weight strategy is employed during training. Experimentally, our proposed network outperforms state-of-the-art methods on a public PASCAL 3D+ dataset, which verifies the effectiveness of our method and further corroborates the above proposition.

Object pose estimation from a single image is a fundamental and challenging problem in computer vision and robotics. Generally, current methods treat pose estimation as a classification or a regression problem. However, regression based methods usually suffer from the issue of imbalanced training data, while classification methods are difficult to discriminate nearby poses. In this paper, a hybrid CNN model, which we call it HCR-Net that integrates both a classification network and a regression network, is proposed to deal with these issues. Our model is inspired by that regression methods can get better accuracy on homogeneously distributed datasets while classification methods are more effective for coarse quantization of the poses even if the dataset is not well balanced. The classification methods and the regression methods essentially complement each other. Thus we integrate both them into a neural network in a hybrid fashion and train it end-to-end with two novel loss functions. As a result, our method surpass the state-of-the-art methods, even with imbalanced training data and much less data augmentation. The experimental results on the challenging Pascal3D+ database demonstrate that our method outperforms the state-of-the-arts significantly, achieving improvements on ACC and AVP metrics up to 4% and 6%, respectively.

A three-dimensional (3D) mesh compression algorithm based on novel prediction methods and a mode decision scheme is proposed in this work. After decomposing an input mesh into base and refinement layers, we segment the geometry data of each layer into clusters. To encode vertex positions efficiently, we propose two prediction methods: the dual ring prediction and the minimum mean square error (MMSE) prediction. Also, we develop a mode decision scheme that selects the best prediction mode for each cluster. Simulation results demonstrate that the proposed algorithm provides significantly better compression performance than conventional techniques.

In this work, we propose a progressive compression algorithm using topology-based Karhunen-Loeve transform(KLT). First, we simplify an input mesh to represents an original mesh in several level of details. Then, coordinates of decimated vertices at each level are predicted from the coarser level mesh, and the prediction residuals are transmitted to a decoder. To provide high coding efficiency, we apply the topology-based KLT, which compacts the energy into a few coefficients, to the prediction residuals. Moreover, we develop a bit plane coder, which uses a context-adaptive arithmetic coder, for the entropy coding. Experiments on various 3D meshes show that the proposed algorithm provides enhanced compression performance.

We propose a novel technique for extracting a series of displaced subdivision surfaces sharing the same topology and the same displacement map from a given animated mesh. Our motion-based mesh simplification method creates control meshes with a small number of vertices but keeps the motion information. Extracted control meshes are simpler than the original meshes, and so easier to edit and take less storage. Our method uses only one displacement map for all frames, which greatly reduces the amount of data.

We propose an out-of-core method for creating semi-regular surface representations from large input surface meshes. Our approach is based on a streaming implementation of the MAPS remesher of Lee et al. [18]. Our remeshing procedure consists of two stages. First, a simplification process is used to obtain the base domain. During simplification, we maintain the mapping information between the input and the simplified meshes. The second stage of remeshing uses the mapping information to produce samples of the output semi-regular mesh. The out-of-core operation of our method is enabled by the synchronous streaming of a simplified mesh and the mapping information stored at the original vertices. The synchronicity of two streaming buffers is maintained using a specially designed write strategy for each buffer. Experimental results demonstrate the remeshing performance of the proposed method, as well as other applications that use the created mapping between the simplified and the original surface representations.

A surface light field is a function that gives the colors of each object point viewed from different directions. Object representation with a surface light field provides a nice structure for 3D photography. This paper presents a feature-based morphing technique for two objects equipped with surface light fields. The technique consists of geometry morphing and in-between light field mapping. Geometry morphing is accomplished by 3D mesh morphing, where we introduce a vertex merging technique to generate a simpler metamesh. In in-between light field mapping, an in-between object is rendered by extracting necessary fragments from input surface light fields. We also propose an acceleration technique for rendering an in-between object. Experimental results with real and synthetic data show natural and plausible morphing between objects with surface light fields. The proposed morphing technique can be used as an editing tool for 3D photography.

3D mesh morphing based on a metamesh has fundamental limitations of complicated in-between meshes and no topology (connectivity) changes in a metamorphosis. This paper presents a novel approach for 3D mesh morphing, which is not based on a metamesh. The approach simultaneously interpolates the topology and geometry of input meshes. In our approach, an in-between mesh contains only the vertices from the source and target meshes. Since no additional vertices are introduced, the in-between meshes are much simpler than those generated by previous techniques.