In this paper we propose a lightweight object detector by optimizing the structure of YOLOv3. The optimization is conducted in two aspects: simplifying the structural components by lightweight substitutes and introducing the attention mechanism to increase the detection accuracy. For the simplification, we remodel the backbone based on MobileNet v2 and replace every 3×3 convolution in the detection neck and head by the fusion of a 3 × 3 depthwise separable convolution and a squeeze and excitation block (DSConv+SE); for the attention enhancement, we introduce the high-frequency wavelets of the original image to the input, a simplified non- local block to the simplified backbone and convolutional block attention modules to the simplified detection neck. In addition, local 3×3 convolution branches are introduced to the simplified backbone for enhanced learning capability. Experiments demonstrate that the proposed detector outperforms each compared state-of-the-art work in one or more aspects.

In this work, we propose LiteMVGNet, a novel lightweight neural network for 3D shape classification. It is based on depth maps generated by multi-view rendering of the corresponding 3D model. LiteMVGNet is designed to be lightweight and effective in various aspects. First, the views and corresponding depth maps are partitioned into groups. Next, depth map features for each group are separately extracted by an adapted MobileNetV2 block. Finally, the extracted group features are fused by an adapted MobileViT block. The views are partitioned by good geometrical semantics and ECAnet is utilized to facilitate extraction of effective features. As demonstrated by experiments, in comparison with the state-of-the-art benchmark models, the proposed one cuts the network parameter count by a third and more and reduces the floating-point operation count by even one or two orders of magnitude. Still, the proposed model yields classification accuracies comparable with the benchmark models.

Human Motion Capture (MoCap) has emerged as the most popular method for human animation production. However, due to joint occlusion, marker shedding, and equipment imprecision, the raw motion data is often corrupted, leading to missing motion data. To address this issue, a missing motion data recovery method utilizing attention-based transformers is proposed in this paper. The proposed model consists of two levels of transformers and a regression head. The first level of transformers extract the spatial features within each frame, and the second level of transformer integrates the per-frame features across time to capture temporal dependencies. The integrated features are then sent to the regression head to derive the complete motion. Extensive experiments on the CMU database demonstrate that the proposed model consistently outperforms the other state-of-the-art methods in recovery accuracy.

Targeting both computing and coding efficiencies, we propose in this work a novel hierarchical bit-wise differential coding scheme to compress point cloud attributes. The encoder firstly quantizes and organizes the points into an octree structure and, for each internal node, picks its attribute(s) from a child named source child. Next, the encoder conducts a top-down scanning of the hierarchy. For each node with more than one child, it computes the bit-wise attribute difference between the current node and each non-source child by exclusive-OR and encodes the difference with an arithmetic coder. Further, a table look-up approach is proposed to accelerate the online source child identification. The proposed scheme produces superior computing and coding efficiencies for lossless point cloud attribute compression, outperforming the MPEG benchmark coders by large margins in our experiments.

In this work, we propose an efficient retrieval method for human motion capture (MoCap) data based on supervised deep hash code learning. Raw Mocap data is represented into three 2D images, which encode the trajectories, velocities and self-similarity of joints respectively. Such image-based representations are fed into a convolutional neural network (CNN) adapted from the pre-trained VGG16 network. Further, we add a hash layer to fine-tune the CNN and generate the hash codes. By minimizing the loss defined by classification error and constraints on hash codes, highly discriminative hash representations of the motion data can be generated. As experimentally demonstrated on the public HDM05 data set, our algorithm achieves high accuracy comparing with the state-of-the-art MoCap data retrieval algorithms. Besides, it achieves high efficiency due to the fast matching of hash codes.

One major branch of saliency object detection methods are diffusion-based which construct a graph model on a given image and diffuse seed saliency values to the whole graph by a diffusion matrix. While their performance is sensitive to specific feature spaces and scales used for the diffusion matrix definition, little work has been published to systematically promote the robustness and accuracy of salient object detection under the generic mechanism of diffusion. In this work, we firstly present a novel view of the working mechanism of the diffusion process based on mathematical analysis, which reveals that the diffusion process is actually computing the similarity of nodes with respect to the seeds based on diffusion maps. Following this analysis, we propose super diffusion, a novel inclusive learning-based framework for salient object detection, which makes the optimum and robust performance by integrating a large pool of feature spaces, scales and even features originally computed for non-diffusion-based salient object detection. A closed-form solution of the optimal parameters for the integration is determined through supervised learning. At the local level, we propose to promote each individual diffusion before the integration. Our mathematical analysis reveals the close relationship between saliency diffusion and spectral clustering. Based on this, we propose to re-synthesize each individual diffusion matrix from the most discriminative eigenvectors and the constant eigenvector (for saliency normalization). The proposed framework is implemented and experimented on prevalently used benchmark datasets, consistently leading to state-of-the-art performance.

In this work, we propose an original scheme for generic content-based retrieval of marker-based motion capture data. It works on motion capture data of arbitrary subject types and arbitrary marker attachment and labelling conventions. Specifically, we propose a novel motion signature to statistically describe both the high-level and the low-level morphological and kinematic characteristics of a motion capture sequence, and conduct the content-based retrieval by computing and ordering the motion signature distance between the query and every item in the database. The distance between two motion signatures is computed by a weighted sum of differences in separate features contained in them. For maximum retrieval performance, we propose a method to pre-learn an optimal set of weights for each type of motion in the database through biased discriminant analysis, and adaptively choose a good set of weights for any given query at the run time. Excellence of the proposed scheme is experimentally demonstrated on various data sets and performance metrics.

In this work, we propose to retrieve images according to a subspace-projected color and texture (SuPCAT) feature descriptor. Firstly, we propose a novel contrast and color distribution (CoCD) descriptor to characterize the pixels' colors in an image. Thereafter, we combine the proposed CoCD color feature descriptor with four effective texture feature descriptors and conduct subspace projection on the combined feature descriptor for reduced dimensionality and increased discriminativeness, which leads to the proposed SuPCAT feature descriptor. By integrating the SuPCAT feature descriptor with a basic Euclidean distance metric, we construct an image retrieval scheme and conduct experiments to demontrate its outstanding performance.

In this work, we propose a generic scheme to promote any diffusion-based salient object detection algorithm by original ways to re-synthesize the diffusion matrix and construct the seed vector. We first make a novel analysis of the working mechanism of the diffusion matrix, which reveals the close relationship between saliency diffusion and spectral clustering. Following this analysis, we propose to re-synthesize the diffusion matrix from the most discriminative eigenvectors after adaptive re-weighting. Further, we propose to generate the seed vector based on the readily available diffusion maps, avoiding extra computation for color-based seed search. As a particular instance, we use inverse normalized Laplacian matrix as the original diffusion matrix and promote the corresponding salient object detection algorithm, which leads to superior performance as experimentally demonstrated.

It is a crucial problem to estimate head pose automatically and robustly in many visual applications. In order to solve this problem, we propose in this work a novel and simple face image descriptor (i.e., block energy map) and, based on which, complete schemes for automatic and robust head pose estimation using support vector regression and Gaussian processes regression, respectively. The proposed descriptor and schemes contrast with many of the previously published ones that rely on manual assistance to locate the face position in an input image and/or are sensitive to factors such as identity and misalignment. Experimental results demonstrate the superiority of the proposed descriptor and schemes.

A novel scheme for single-rate compression of material and texture attributes for triangular meshes is proposed in this work. For the material coding, it proposes a novel approach based on breadth-first surface traversal, exploiting the local coherence of the material attributes among neighboring facets; for the texture coordinate coding, it proposes a similar-triangle-based prediction method, exploiting the correlation between 3D geometry and its texture-space parameterization. As a result, the proposed algorithm achieves efficient coding of the material and the texture attributes, as experimentally demonstrated.

In this work, we propose improvements on a state-of-the-art scheme for progressive triangular mesh compression, which is based on hierarchical vertex clustering. Specifically, we fix several shortcomings of that scheme including quantization error accumulation, false facet generation and lack of facet orientation encoding. In addition, the resultant mesh encoder yields better rate-distortion performance than another state-of-the-art one on our test model, as demonstrated by the experiments.

ABSTRACT In this paper, we propose a novel genetic algorithm approach to human motion capture (MoCap) data segmentation. For a given MoCap sequence, it constructs a symbolic representation through unsupervised sparse learning, detects the candidate segmenting points to the sequence, models the selection/deselection of each candidate with a gene, and employs the genetic algorithm to find the optimal solution. To the best of our knowledge, we for the first time introduce the genetic algorithm and the sparse learning technique to the problem of MoCap data segmentation, leading to excellent segmentation performance as experimentally demonstrated. Copyright © 2014 John Wiley & Sons, Ltd.

We propose in this paper a novel framework for serial multimodal biometric systems based on semisupervised learning techniques. The proposed framework addresses the inherent issues of user inconvenience and system inefficiency in parallel multimodal biometric systems. Further, it advances the serial multimodal biometric systems by promoting the discriminating power of the weaker but more user convenient trait(s) and saving the use of the stronger but less user convenient trait(s) whenever possible. This is in contrast to other existing serial multimodal biometric systems that suggest optimized orderings of the traits deployed and parameterizations of the corresponding matchers but ignore the most important requirements of common applications. In terms of methodology, we propose to use semisupervised learning techniques to strengthen the matcher(s) on the weaker trait(s), utilizing the coupling relationship between the weaker and the stronger traits. A dimensionality reduction method for the weaker trait(s) based on dependence maximization is proposed to achieve this purpose. Experiments on two prototype systems clearly demonstrate the advantages of the proposed framework and methodology.

The goal of saliency detection is to locate important pixels or regions in an image which attract humans' visual attention the most. This is a fundamental task whose output may serve as the basis for further computer vision tasks like segmentation, resizing, tracking and so forth. In this paper we propose a novel salient region detection algorithm by integrating three important visual cues namely uniqueness, focus ness and objectness (UFO). In particular, uniqueness captures the appearance-derived visual contrast, focus ness reflects the fact that salient regions are often photographed in focus, and objectness helps keep completeness of detected salient regions. While uniqueness has been used for saliency detection for long, it is new to integrate focus ness and objectness for this purpose. In fact, focus ness and objectness both provide important saliency information complementary of uniqueness. In our experiments using public benchmark datasets, we show that, even with a simple pixel level combination of the three components, the proposed approach yields significant improvement compared with previously reported methods.

To diagnose the patella dislocation, various angles and distances need to be measured on knee CT images, which was traditionally done by doctors manually. In this work, we propose a novel scheme for automatic measurement on knee CT images to assist doctors diagnosis of patella dislocation. Specifically, we first segment the femur and the patella regions on the CT images, then adopt optimal fitting to obtain the central planes of the femur and the patella bones and, based on which, make the measurement. As experimentally demonstrated, the measured results obtained with our system are highly consistent with those manually made by experienced doctors.

In this paper, we present a novel method for realtime synthesis of human motion under external perturbations. The proposed method is data-driven and based on angular momentum analysis. When an external force is applied on the virtual human body, we analyze the change in the joints' angular momentums in a short period of time, predict the human body response, find an appropriate motion sequence from the pre-built motion capture (MoCap) database, and make a smooth transition between the current and the retrieved motion sequences to obtain the synthesized motion. The most important contributions of our method include that we propose a complete momentum analysis solution for the human body and that we make effective MoCap data organization based on the major characteristics of the body motion and the external force. As a result, realistic and real-time human motion synthesis is achieved, as experimentally demonstrated with the walking, the running and the jumping sequences.

Abstract A feature‐oriented generic progressive lossless mesh coder (FOLProM) is proposed to encode triangular meshes with arbitrarily complex geometry and topology. In this work, a sequence of levels of detail (LODs) are generated through iterative vertex set split and bounding volume subdivision. The incremental geometry and connectivity updates associated with each vertex set split and/or bounding volume subdivision are entropy coded. Due to the visual importance of sharp geometric features, the whole geometry coding process is optimized for a better presentation of geometric features, especially at low coding bitrates. Feature‐oriented optimization in FOLProM is performed in hierarchy control and adaptive quantization. Efficient coordinate representation and prediction schemes are employed to reduce the entropy of data significantly. Furthermore, a simple yet efficient connectivity coding scheme is proposed. It is shown that FOLProM offers a significant rate‐distortion (R‐D) gain over the prior art, which is especially obvious at low bitrates.

Abstract In this work, a novel scheme is proposed to compress human motion capture data based on hierarchical structure construction and motion pattern indexing. For a given sequence of 3D motion capture data of human body, the 3D markers are first organized into a hierarchy where each node corresponds to a meaningful part of the human body. Then, the motion sequence corresponding to each body part is coded separately. Based on the observation that there is a high degree of spatial and temporal correlation among the 3D marker positions, we strive to identify motion patterns that form a database for each meaningful body part. Thereafter, a sequence of motion capture data can be efficiently represented as a series of motion pattern indices. As a result, higher compression ratio has been achieved when compared with the prior art, especially for long sequences of motion capture data with repetitive motion styles. Another distinction of this work is that it provides means for flexible and intuitive global and local distortion controls.

In this paper, we propose a generic point cloud encoder that provides a unified framework for compressing different attributes of point samples corresponding to 3D objects with arbitrary topology. In the proposed scheme, the coding process is led by an iterative octree cell subdivision of the object space. At each level of subdivision, positions of point samples are approximated by the geometry centers of all tree-front cells while normals and colors are approximated by their statistical average within each of tree-front cells. With this framework, we employ attribute-dependent encoding techniques to exploit different characteristics of various attributes. All of these have led to significant improvement in the rate-distortion (R-D) performance and a computational advantage over the state of the art. Furthermore, given sufficient levels of octree expansion, normal space partitioning and resolution of color quantization, the proposed point cloud encoder can be potentially used for lossless coding of 3D point clouds.

A view-dependent visibility estimation technique for tree models is proposed in this work. While most previous work focused on visibility estimation of large objects in architectural walkthroughs, we consider the visibility estimation problem of tree leaves in complex natural scenes such as forests. Exact calculation of how each leaf object blocks another from a given viewpoint is difficult due to the large amount of computational cost. To address this issue, we propose three simple yet effective methods for efficient visibility estimation of tree leaves: the distance-based, the normal-based, and the layer-plus-normal-based methods. Experimental results show that the distance-based method always produces the best analytical performance, the normal-based method works better in preserving volumetric visual quality at low budget constraints, and the layer-plus-normal-based method preserves volumetric visual quality while yielding excellent analytical performance.

A new progressive lossless 3D triangular mesh encoder is proposed in this work, which can encode any 3D triangular mesh with an arbitrary topological structure. Given a mesh, the quantized 3D vertices are first partitioned into an octree (OT) structure, which is then traversed from the root and gradually to the leaves. During the traversal, each 3D cell in the tree front is subdivided into eight childcells. For each cell subdivision, both local geometry and connectivity changes are encoded, where the connectivity coding is guided by the geometry coding. Furthermore, prioritized cell subdivision is performed in the tree front to provide better rate-distortion (RD) performance. Experiments show that the proposed mesh coder outperforms the kd-tree algorithm in both geometry and connectivity coding efficiency. For the geometry coding part, the range of improvement is typically around 10%~20%, but may go up to 50%~60% for meshes with highly regular geometry data and/or tight clustering of vertices.

A progressive 3D geometry coding scheme using octree-based space partitioning is proposed in this work, which achieves better coding efficiency than the state-of-the-art geometric codec known as the kd-tree-based codec. Given a 3D mesh, the quantized 3D vertices are first partitioned into an octree structure. The octree is then traversed from the root and gradually to the leaves and, during the traversal, each 3D cell in the tree front is subdivided along three orthogonal directions. For each cell subdivision, an 8-bit bitpattern is generated, reordered, and entropy encoded. Furthermore, selective cell subdivision is performed to provide better rate-distortion performance, especially at low bitrates. It is shown in experimental results that the coding cost is around 5.5 bits per vertex (bpv) for 8-bit coordinate quantization and 16.6 bpv for 12-bit coordinate quantization on the average. The rate-distortion performance of the proposed algorithm is significantly better than that of the kd-tree-based codec, especially at low bitrates.

A progressive 3D geometry coding scheme based on the octree structure is proposed in this work, which achieves better coding efficiency than the state-of-the-art geometric codec known as the kd-tree-based codec. Given a 3D mesh, the quantized 3D vertices are first partitioned into an octree structure. The octree is then traversed from the root and gradually to the leaves and, during the traversal, each 3D cell in the tree front is subdivided along three orthogonal directions. For each cell subdivision, the order of subdivision directions is adaptively chosen, the neighborhood-prediction is used and the vertex number distribution is efficiently encoded. The final output bit stream contains relevant information associated with each cell subdivision. It is shown in experimental results that the coding cost is around 15.6 bits per vertex (bpv) for 12-bit coordinate quantization and 5.6 bpv for 8-bit coordinate quantization on average.