Time-varying meshes (TVMs), characterized by their varying connectivity and number of vertices, hold significant potential in AR/VR applications. However, their practical use is challenging due to their large file sizes and the complexity of time-varying topology. Many time-varying mesh compression methods attempted to exploit redundancy between consecutive meshes to compress TVMs more efficiently, however, most face difficulties in establishing stable vertex and surface correspondence between the frames of a TVM. We propose TVMC, a novel TVM compression method that leverages volume tracking and extracts high-quality reference meshes for inter-frame prediction. Specifically, we use as-rigid-as-possible volume tracking to align consecutive TVMs and track volume centers, followed by multidimensional scaling to refine reference centers. This allows us to precisely deform a group of frames to the reference space and extract the reference mesh which is then deformed to approximate each mesh in the group to get displacement fields for TVM compression. Extensive experiments show that TVMC outperforms state-of-the-art methods (e.g., Google Draco, V-DMC 4.0, etc.), with bitrates of 4-6 Mbps compared to 9--12 Mbps for Draco and 10-15 Mbps for V-DMC 4.0. It reduces the decoding time by 66.1% compared to Draco and enables an increased group of frames (up to 15) without significant distortion.

Abstract Time‐varying meshes (TVMs), that is mesh sequences with varying connectivity, are a greatly versatile representation of shapes evolving in time, as they allow a surface topology to change or details to appear or disappear at any time during the sequence. This, however, comes at the cost of large storage size. Since 2003, there have been attempts to compress such data efficiently. While the problem may seem trivial at first sight, considering the strong temporal coherence of shapes represented by the individual frames, it turns out that the varying connectivity and the absence of implicit correspondence information that stems from it makes it rather difficult to exploit the redundancies present in the data. Therefore, efficient and general TVM compression is still considered an open problem. We describe and categorize existing approaches while pointing out the current challenges in the field and hint at some related techniques that might be helpful in addressing them. We also provide an overview of the reported performance of the discussed methods and a list of datasets that are publicly available for experiments. Finally, we also discuss potential future trends in the field.

The paper introduces a new method to describe the surfaces of voxelised solids. It operates in three stages: a hierarchical linked list of chain code sequences is created first; the linked lists are pruned; and, finally, the content of the data structure is stored. The method uses chain codes from either a three- or nine-symbols alphabet. In the first case, two chain code symbols are needed to access the next face, while, in the second case, this is done by one symbol. The pair of chain codes from the three-symbols alphabet, or the individual symbol from the nine-symbols alphabet are considered as tokens. The sets of tokens are, in both cases, extended by two tokens, indicating the beginning and ending of the list. The method processes solids of any shape, including those containing holes, cavities, or multiple components existing in the same voxel space. Edge-connectivity is permitted. The method was compared against the method proposed by Lemus et al., which is designed for solids without holes. Although supporting a broader set of voxelised solids, the proposed method generates sequences of tokens that are, on average, up to 10% shorter. Since the information entropy of the sequences of tokens produced by the proposed method is also smaller, the obtained sequences are more compressible, as confirmed by applying gzip and bzip2 data compressors.

Abstract In certain practical situations, the connectivity of a triangle mesh needs to be transmitted or stored given a fixed set of 3D vertices that is known at both ends of the transaction (encoder/decoder). This task is different from a typical mesh compression scenario, in which the connectivity and geometry (vertex positions) are encoded either simultaneously or in reversed order (connectivity first), usually exploiting the freedom in vertex/triangle re‐indexation. Previously proposed algorithms for encoding the connectivity for a known geometry were based on a canonical mesh traversal and predicting which vertex is to be connected to the part of the mesh that is already processed. In this paper, we take this scheme a step further by replacing the fixed traversal with a priority queue of open expansion gates, out of which in each step a gate is selected that has the most certain prediction, that is one in which there is a candidate vertex that exhibits the largest advantage in comparison with other possible candidates, according to a carefully designed quality metric. Numerical experiments demonstrate that this improvement leads to a substantial reduction in the required data rate in comparison with the state of the art.

The new generation 3D scanner devices have revolutionized the way information from 3D objects is acquired, making the process of scene capturing and digitization straightforward. However, the effectiveness and robustness of conventional algorithms for real scene analysis are usually deteriorated due to challenging conditions, such as noise, low resolution, and bad perceptual quality. In this work, we present a methodology for identifying and registering partially-scanned and noisy 3D objects, lying in arbitrary positions in a 3D scene, with corresponding high-quality models. The methodology is assessed on point cloud scenes with multiple objects with large missing parts. The proposed approach does not require connectivity information and is thus generic and computationally efficient, thereby facilitating computationally demanding applications, like augmented reality. The main contributions of this work are the introduction of a layered joint registration and indexing scheme of cluttered partial point clouds using a novel multi-scale saliency extraction technique to identify distinctive regions, and an enhanced similarity criterion for object-to-model matching. The processing time of the process is also accelerated through 3D scene segmentation. Comparisons of the proposed methodology with other state-of-the-art approaches highlight its superiority under challenging conditions.

Abstract Random Sample Consensus is a powerful paradigm that was successfully applied in various contexts, including Location Determination Problem, fundamental matrix estimation and global 3D surface registration, where many previously proposed algorithms can be interpreted as a particular implementation of this concept. In general, a set of candidate transformations is generated by some simple procedure, and an aligning transformation is chosen within this set, such that it aligns the largest portion of the input data. We observe that choosing the aligning transformation may also be interpreted as finding consensus among the candidates, which in turn involves measuring similarity of candidate rigid transformations. While it is not difficult to construct a metric that provides reasonable results, most approaches come with certain limitations and drawbacks. In this paper, we investigate possible means of measuring distances in SE(3) and compare their properties both theoretically and experimentally in a model RANSAC registration algorithm. We also propose modifications to existing measures and propose a novel method of locating the consensus transformation based on Vantage Point Tree data structure.

Abstract Laplacian mesh compression, also known as high‐pass mesh coding, is a popular technique for efficiently storing both static and dynamic triangle meshes that gained further recognition with the advent of perceptual mesh distortion evaluation metrics. Currently, the usual rule of thumb that drives the decision for a mesh compression algorithm is whether or not accuracy in absolute scale is required: Laplacian mesh encoding is chosen when perceptual quality is the main objective, while other techniques provide better results in terms of mechanistic error measures such as mean squared error. In this work, we present a modification of the Laplacian mesh encoding algorithm that preserves its benefits while it substantially reduces the resulting absolute error. Our approach is based on analyzing the reconstruction stage and modifying the quantization of differential coordinates, so that the decoded result stays close to the input even in areas that are distant from anchor points. In our approach, we avoid solving an overdetermined system of linear equations and thus reduce data redundancy, improve conditioning and achieve faster processing. Our approach can be directly applied to both static and dynamic mesh compression and we provide quantitative results comparing our approach with the state of the art methods.

Abstract While it is usually not difficult to compute principal curvatures of a smooth surface of sufficient differentiability, it is a rather difficult task when only a polygonal approximation of the surface is available, because of the inherent ambiguity of such representation. A number of different approaches has been proposed in the past that tackle this problem using various techniques. Most papers tend to focus on a particular method, while an comprehensive comparison of the different approaches is usually missing. We present results of a large experiment, involving both common and recently proposed curvature estimation techniques, applied to triangle meshes of varying properties. It turns out that none of the approaches provides reliable results under all circumstances. Motivated by this observation, we investigate mesh statistics, which can be computed from vertex positions and mesh connectivity information only, and which can help in deciding which estimator will work best for a particular case. Finally, we propose a meta‐estimator, which makes a choice between existing algorithms based on the value of the mesh statistics, and we demonstrate that such meta‐estimator, despite its simplicity, provides considerably more robust results than any existing approach.

3D meshes are deployed in a wide range of application processes (e.g., transmission, compression, simplification, watermarking and so on) which inevitably introduce geometric distortions that may alter the visual quality of the rendered data. Hence, efficient model-based perceptual metrics, operating on the geometry of the meshes being compared, have been recently introduced to control and predict these visual artifacts. However, since the 3D models are ultimately visualized on 2D screens, it seems legitimate to use images of the models (i.e., snapshots from different viewpoints) to evaluate their visual fidelity. In this work we investigate the use of image metrics to assess the visual quality of 3D models. For this goal, we conduct a wide-ranging study involving several 2D metrics, rendering algorithms, lighting conditions and pooling algorithms, as well as several mean opinion score databases. The collected data allow (1) to determine the best set of parameters to use for this image-based quality assessment approach and (2) to compare this approach to the best performing model-based metrics and determine for which use-case they are respectively adapted. We conclude by exploring several applications that illustrate the benefits of image-based quality assessment.

Abstract Lossy compression of motion capture data can alleviate the problems of efficient storage and transmission by exploiting the redundancy and the superfluous precision of the data. When considering the acceptable amount of distortion, perceptual issues have to be taken into account. Current state of the art methods reduce the data rate required for high quality storage of motion capture data using various techniques. Most of them, however, do not use the common tools of general data compression, such as the method of Lagrange multipliers, and thus they obtain sub‐optimal results, making it difficult to do a fair comparison of their performance. In this paper, we present a general preprocessing step based on Lagrange multipliers, which allows to rigorously adjust the precision in each of the degrees of freedom of the input data according to the amount of influence the given degree of freedom has on the overall distortion. We then present a simple compression method based on Principal Component Analysis, which in combination with the proposed preprocessing achieves significantly better results than current state of the art methods. It allows optimization with respect to various distortion metrics, and we discuss the choice of the metric in two common but distinct scenarios, proposing a perceptually oriented comparison metric based on the relation of the problem at hand to the problem of compression of dynamic meshes.

Abstract In this paper, we investigate the possibilities of efficient encoding of UV coordinates associated with vertices of a triangle mesh. Since most parametrization schemes attempt to achieve at least some level of conformality, we exploit the similarity of the shapes of triangles in the mesh and in the parametrization. We propose two approaches building on this idea: first, applying a recently proposed generalization of the parallelogram predictor, using the inner angles of mesh triangles corresponding to the UV‐space triangles. Second, we propose an encoding method based on discrete Laplace operator, which also allows exploiting the information contained in the mesh geometry to efficiently encode the parametrization. Our experiments show that the proposed approach leads to savings of up to 3 bits per UV vertex, without loss of precision.

Abstract This paper addresses the problem of representing dynamic 3D meshes in a compact way, so that they can be stored and transmitted efficiently. We focus on sequences of triangle meshes with shared connectivity, avoiding the necessity of having a skinning structure. Our method first computes an average mesh of the whole sequence in edge shape space. A discrete geometric Laplacian of this average surface is then used to encode the coefficients that describe the trajectories of the mesh vertices. Optionally, a novel spatio‐temporal predictor may be applied to the trajectories to further improve the compression rate. We demonstrate that our approach outperforms the current state of the art in terms of low data rate at a given perceived distortion, as measured by the STED and KG error metrics.

Abstract Almost all mesh processing procedures cause some more or less visible changes in the appearance of objects represented by polygonal meshes. In many cases, such as mesh watermarking, simplification or lossy compression, the objective is to make the change in appearance negligible, or as small as possible, given some other constraints. Measuring the amount of distortion requires taking into account the final purpose of the data. In many applications, the final consumer of the data is a human observer, and therefore the perceptibility of the introduced appearance change by a human observer should be the criterion that is taken into account when designing and configuring the processing algorithms. In this review, we discuss the existing comparison metrics for static and dynamic (animated) triangle meshes. We describe the concepts used in perception‐oriented metrics used for 2D image comparison, and we show how these concepts are employed in existing 3D mesh metrics. We describe the character of subjective data used for evaluation of mesh metrics and provide comparison results identifying the advantages and drawbacks of each method. Finally, we also discuss employing the perception‐correlated metrics in perception‐oriented mesh processing algorithms.

Many algorithms have been proposed for the task of efficient compression of triangular meshes. Geometric properties of the input data are usually exploited to obtain an accurate prediction of the data at the decoder. Considerations on how to improve the prediction usually focus on its normal part, assuming that the tangential part behaves similarly. In this paper, we show that knowledge of vertex valences might allow the decoder to form a prediction that is more accurate in the tangential direction, using a weighted parallelogram prediction. This idea can be easily implemented into existing compression algorithms, such as Edgebreaker, and it can be applied at different levels of sophistication, from very simple ones, that are computationally very cheap, to some more complex ones that provide an even better compression efficiency.

Abstract In computer graphics, triangle meshes are ubiquitous as a representation of surface models. Processing of this kind of data, such as compression or watermarking, often involves an unwanted distortion of the surface geometry. Advanced processing algorithms are continuously being proposed, aiming at improving performance (compression ratio, watermark robustness and capacity), while minimizing the introduced distortion. In most cases, the final resulting mesh is intended to be viewed by a human observer, and it is therefore necessary to minimise the amount of distortion perceived by the human visual system. However, only recently there have been studies published on subjective experiments in this field, showing that previously used objective error measures exhibit rather poor correlation with the results of subjective experiments. In this paper, we present results of our own large subjective testing aimed at human perception of triangle mesh distortion. We provide an independent confirmation of the previous result by Lavoué et al. that most current metrics perform poorly, with the exception of the MSDM/MSDM2 metrics. We propose a novel metric based on measuring the distortion of dihedral angles, which provides even higher correlation with the results of our experiments and experiments performed by other researchers. Our metric is about two orders of magnitude faster than MSDM/MSDM2, which makes it much more suitable for usage in iterative optimisation algorithms.

Abstract Development of geometry data compression techniques in the past years has been limited by the lack of a metric with proven correlation with human perception of mesh distortion. Many algorithms have been proposed, but usually the aim has been to minimise mean squared error, or some of its derivatives. In the field of dynamic mesh compression, the situation has changed with the recent proposal of the STED metric, which has been shown to capture the human perception of mesh distortion much better than previous metrics. In this paper we show how existing algorithms can be steered to provide optimal results with respect to this metric, and we propose a novel dynamic mesh compression algorithm, based on trajectory space PCA and Laplacian coordinates, specifically designed to minimise the newly proposed STED error. Our experiments show that using the proposed algorithm, we were able to reduce the required data rate by up to 50% while preserving the introduced STED error.

There are multiple areas of computer graphics where triangular meshes are being altered in order to reduce their size or complexity, while attempting to preserve the original shape of the mesh as closely as possible. Recently, this area of research has been extended to cover even a dynamic case, i.e., surface animations which are compressed and simplified. However, to date very little effort has been made to develop methods for evaluating the results, namely the amount of distortion introduced by the processing. Even the most sophisticated compression methods use distortion evaluation by some kind of mean squared error while the actual relevance of such measure has not been verified so far. In this paper, we point out some serious drawbacks of the existing error measures. We present results of the subjective testing that we have performed, and we derive a new measure called Spatiotemporal edge difference (STED) which is shown to provide much better correlation with subjective opinions on mesh distortion.

Abstract The task of dynamic mesh compression seeks to find a compact representation of a surface animation, while the artifacts introduced by the representation are as small as possible. In this paper, we present two geometric predictors, which are suitable for PCA‐based compression schemes. The predictors exploit the knowledge about the geometrical meaning of the data, which allows a more accurate prediction, and thus a more compact representation. We also provide rate/distortion curves showing that our approach outperforms the current PCA‐based compression methods by more than 20%.

Abstract In this paper, we present an extension of dynamic mesh compression techniques based on PCA. Such representation allows very compact representation of moving 3D surfaces; however, it requires some side information to be transmitted along with the main data. The biggest part of this information is the PCA basis, and since the data can be encoded very efficiently, the size of the basis cannot be neglected when considering the overall performance of a compression algorithm. We present a new work in this area, as none of the papers about PCA based compression really addresses this issue. We will show that for an efficient and accurate encoding there are better choices than even sophisticated algorithms such as LPC. We will present results showing that our approach can reduce the size of the basis by 90% with respect to direct encoding, which can lead to approximately 25% increase of performance of the compression algorithm without any significant loss of accuracy. Such improvement moves the performance of the PCA encoder beyond the performance of current state of the art dynamic mesh compression algorithms, such as the recently adopted MPEG standard, FAMC.

Abstract We present a new approach to dynamic mesh compression, which combines compression with simplification to achieve improved compression results, a natural support for incremental transmission and level of detail. The algorithm allows fast progressive transmission of dynamic 3D content. Our scheme exploits both temporal and spatial coherency of the input data, and is especially efficient for the case of highly detailed dynamic meshes. The algorithm can be seen as an ultimate extension of the clustering and local coordinate frame (LCF)‐based approaches, where each vertex is expressed within its own specific coordinate system. The presented results show that we have achieved better compression efficiency compared to the state of the art methods. Copyright © 2008 John Wiley & Sons, Ltd.