The evaluation of perceptual quality in 3D mesh compression, particularly for Video-based Dynamic Mesh Coding (V-DMC), is challenged by the scarcity of subject-rated datasets and the high computational cost of full-mesh decoding and the sophisticated visual feature extraction steps. To bridge this gap, we first introduce a novel V-DMC distortion dataset, comprising 16 high-quality original meshes and 400 compressed, textureless variants. We conducted a subjective quality assessment study with 30 participants using the Double Stimulus Impairment Scale (DSIS) method to collect reliable Mean Opinion Scores (MOS). We then propose streamMQ, the first-of-its-kind no-reference, bitstream-layer model for perceptual quality assessment of V-DMC compressed meshes. By extracting key geometric features such as quantization parameters and triangle count directly from the compressed bitstream, streamMQ predicts perceptual quality without full decoding. Experimental evaluation and comparison with state-of-the-art methods demonstrate that streamMQ achieves highly competitive quality assessment performance at tiny fractions of computational and storage costs, facilitating real-time and low-storage application environments. The dataset and source code will be made publicly available at https://github.com/HFL01/QDU-GDM.

No-reference bitstream-layer point cloud quality assessment (PCQA) can be deployed without full decoding at any network node to achieve real-time quality monitoring. In this work, we develop the first PCQA model dedicated to Trisoup-Lifting encoded 3D point clouds by analyzing bitstreams without full decoding. Specifically, we investigate the relationship among texture bitrate per point (TBPP), texture complexity (TC) and texture quantization parameter (TQP) while geometry encoding is lossless. Subsequently, we estimate TC by utilizing TQP and TBPP. Then, we establish a texture distortion evaluation model based on TC, TBPP and TQP. Ultimately, by integrating this texture distortion model with a geometry attenuation factor, a function of trisoupNodeSizeLog2 (tNSL), we acquire a comprehensive NR bitstream-layer PCQA model named streamPCQ-TL. In addition, this work establishes a database named WPC6.0, the first PCQA database dedicated to Trisoup-Lifting encoding mode, encompassing 400 distorted point clouds with 4 geometry multiplied by 5 texture distortion levels. Experiment results on M-PCCD, ICIP2020 and the proposed WPC6.0 database suggest that the proposed streamPCQ-TL model exhibits robust and notable performance in contrast to existing advanced PCQA metrics, particularly in terms of computational cost.

Due to the non-uniform perception of human vision, structural or color changes in salient regions play a dominant role in point cloud quality assessment (PCQA). In this paper, we propose a perception-weighted multi-view PCQA method based on saliency-guided coverage analysis (PW-SCQA), which dynamically quantifies the contribution of different viewpoints on perceptual quality through saliency. First, multi-view projection images are generated based on a polyhedral projection mechanism and multi-scale features are extracted for constructing a 2D saliency map. Then, the 2D to 3D saliency propagation model is used to refine the point-level saliency weights and achieve point cloud saliency visualization. Subsequently, a perception-driven viewpoint optimization mechanism and a novel viewpoint saliency region coverage (SRC) index are innovatively introduced, in which the viewpoint evaluation weights are dynamically adjusted by calculating the SRC of high, medium, and low saliency under the candidate viewpoints. Finally, the multi-view information content weighting image quality assessment method is combined to predict the overall point cloud quality. PW-SCQA outperforms several state-of-the-art methods on three different PCQA datasets.

With the rapid development of immersive multimedia technology, the growing demand for high-quality visual experiences has driven the emergence of point cloud quality assessment (PCQA). While current deep learning-based PCQA models have achieved breakthroughs in performance, problems such as high computational complexity and limited model generalization ability still need to be solved. In this study, focusing on compression distortion, we analyzed and verified that the compression quantization parameter (QP) can be used as a key feature for predicting perceptual quality. Based on this, a novel no-reference point cloud perceptual quality assessment metric, DQP-PCQA, is proposed. Unlike existing PCQA models that only use mean opinion score (MOS) as a supervisory label, this study proposes a multi-objective constrained optimization scheme that adds geometric quantization parameter (GQP) and texture quantization parameter (TQP) as auxiliary supervisory labels to help the model can learn robust perceptual features that take into account both subjective quality and objective distortion. We conducted comparative experiments with other advanced PCQA models on several mainstream PCQA datasets. The results show that the DQP-PCQA model achieves fast convergence speed, excellent and stable performance, low complexity and strong generalization. Further migration experiments show that after applying our proposed method to other advanced PCQA models, the performance of the improved model is further improved. Our discovery provides new insight for PCQA research. To facilitate future reproducible research, the source code will be publicly released at https://github.com/Dds46/DQP-PCQA.

Point cloud compression significantly reduces data volume but sacrifices reconstruction quality, highlighting the need for advanced quality enhancement techniques. Most existing approaches focus primarily on point-to-point fidelity, often neglecting the importance of perceptual quality as interpreted by the human visual system. To address this issue, we propose a generative adversarial network for point cloud quality enhancement (PCE-GAN), grounded in optimal transport theory, with the goal of simultaneously optimizing both data fidelity and perceptual quality. The generator consists of a local feature extraction (LFE) unit, a global spatial correlation (GSC) unit and a feature squeeze unit. The LFE unit uses dynamic graph construction and a graph attention mechanism to efficiently extract local features, placing greater emphasis on points with severe distortion. The GSC unit uses the geometry information of neighboring patches to construct an extended local neighborhood and introduces a transformer-style structure to capture long-range global correlations. The discriminator computes the deviation between the probability distributions of the enhanced point cloud and the original point cloud, guiding the generator to achieve high quality reconstruction. Experimental results show that the proposed method achieves state-of-the-art performance. Specifically, when applying PCE-GAN to the latest geometry-based point cloud compression (G-PCC) test model, it achieves an average BD-rate of -19.2% compared with the PredLift coding configuration and -18.3% compared with the RAHT coding configuration. Subjective comparisons show a significant improvement in texture clarity and color transitions, revealing finer details and more natural color gradients.

The scope of point cloud (PC) applications is expanding. We propose a no-reference bitstream-layer quality assessment model that eliminates the need for full decoding of the PC, providing quality evaluation scores during the V-PCC decoding process. Specifically, we illustrate the relationship between content diversity (CD) and perceptual coding distortion in lossless geometric coding. Subsequently, we model attribute distortion by predicting CD using transform energy (TE) and texture quantization parameter (TQP). By combining the geometric distortion model with geometry quantization parameters (GQP) and the attribute distortion model, we derive comprehensive quality prediction results. Our experimental results on four PC databases (WPC2.0, M-PCCD, VSENSE VVDB and VSENSE VVDB2) show that the proposed energy-adaptive bitstream-layer model (EABL) delivers competitive quality prediction performance in comparison with existing full-reference, reduced-reference and no-reference PC quality assessment models that require full decoding, and meanwhile exhibits large speed advantage. The source code will be made publicly available for repeatability research at https://github.com/arthas-sws/EABL_model.

No-reference point cloud quality assessment (PCQA) based on bitstreams uses information extracted from the bitstream for quality monitoring at network nodes. We develop a no-reference PCQA model based on bitstreams for the perceived quality assessment of Octree-Lifting coded point clouds. At first, our research explores the essential correlation between subjective visual quality degradation and the texture quantization parameter (TQP) when using lossless geometric coding. Then, we enhance the proposed model by incorporating texture complexity (TC) while taking into account the dependence of perceptual coding distortion on the texture characteristics of a point cloud. We estimate TC by utilizing TQP and calculating the average standard deviation of the Y-component of the attribute value ($Y\_ {std}$Y_std), both of which are extracted from the bitstream. Then, a texture distortion assessment model is constructed based on TQP and $Y\_ {std}$Y_std. The integration of the texture distortion model with the position quantization scale (PQS) results in the derivation of an overall no-reference bitstream-based PCQA model, named streamPCQ-OL. The findings from the conducted experiments highlight a significant superiority of the proposed model over existing approaches in terms of performance.

Point cloud perceptual quality assessment plays a critical role in many applications, including compression and communication. We propose PKT-PCQA, a point-based no-reference point cloud quality assessment deep learning network that emulates the human visual system by using progressive knowledge transfer to convert coarse-grained quality classification knowledge into a fine-grained quality prediction task. PKT-PCQA exploits local and global features, as well as an attention mechanism based on spatial and channel attention modules. Experiments on three large and independent point cloud assessment datasets show that PKT-PCQA outperforms existing no-reference and reduced-reference point cloud quality assessment methods and achieves better or similar performance compared to several State-of-the-Art full-reference methods.

We present PU-Mask, a virtual mask-based network for 3D point cloud upsampling. Unlike existing upsampling methods, which treat point cloud upsampling as an “unconstrained generative” problem, we propose to address it from the perspective of “local filling”, i.e., we assume that the sparse input point cloud (i.e., the unmasked point set) is obtained by locally masking the original dense point cloud with virtual masks. Therefore, given the unmasked point set and virtual masks, our goal is to fill the point set hidden by the virtual masks. Specifically, because the masks do not actually exist, we first locate and form each virtual mask by a virtual mask generation module. Then, we propose a mask-guided transformer-style asymmetric auto-encoder (MTAA) to restore the upsampled features. Moreover, we introduce a second-order unfolding attention mechanism to enhance the interaction between the feature channels of MTAA. Next, we generate a coarse upsampled point cloud using a pooling technique that is specific to the virtual masks. Finally, we design a learnable pseudo Laplacian operator to calibrate the coarse upsampled point cloud and generate a refined upsampled point cloud. Extensive experiments demonstrate that PU-Mask is superior to the state-of-the-art methods. Our code will be made available at: https://github.com/liuhaoyun/PU-Mask.

Video-based point cloud compression (V-PCC) is a state-of-the-art moving picture experts group (MPEG) standard for point cloud compression. V-PCC can be used to compress both static and dynamic point clouds in a lossless, near lossless, or lossy way. Many objective quality metrics have been proposed for distorted point clouds. Most of these metrics are full-reference metrics that require both the original point cloud and the distorted one. However, in some real-time applications, the original point cloud is not available, and no-reference or reduced-reference quality metrics are needed. Three main challenges in the design of a reduced-reference quality metric are how to build a set of features that characterize the visual quality of the distorted point cloud, how to select the most effective features from this set, and how to map the selected features to a perceptual quality score. We address the first challenge by proposing a comprehensive set of features consisting of compression, geometry, normal, curvature, and luminance features. To deal with the second challenge, we use the least absolute shrinkage and selection operator (LASSO) method, which is a variable selection method for regression problems. Finally, we map the selected features to the mean opinion score in a nonlinear space. Although we have used only 19 features in our current implementation, our metric is flexible enough to allow any number of features, including future more effective ones. Experimental results on the Waterloo point cloud dataset version 2 (WPC2.0) and the MPEG point cloud compression dataset (M-PCCD) show that our method, namely PCQAML, outperforms state-of-the-art full-reference and reduced-reference quality metrics in terms of Pearson linear correlation coefficient, Spearman rank order correlation coefficient, Kendall's rank-order correlation coefficient, and root mean squared error.

With the increasing demand of compressing and streaming 3D point clouds under constrained bandwidth, it has become ever more important to accurately and efficiently determine the quality of compressed point clouds, so as to assess and optimize the quality-of-experience (QoE) of end users. Here we make one of the first attempts developing a bitstream-based no-reference (NR) model for perceptual quality assessment of point clouds without resorting to full decoding of the compressed data stream. Specifically, we first establish a relationship between texture complexity and the bitrate and texture quantization parameters based on an empirical rate-distortion model. We then construct a texture distortion assessment model upon texture complexity and quantization parameters. By combining this texture distortion model with a geometric distortion model derived from Trisoup geometry encoding parameters, we obtain an overall bitstream-based NR point cloud quality model named streamPCQ. Experimental results show that the proposed streamPCQ model demonstrates highly competitive performance when compared with existing classic full-reference (FR) and reduced-reference (RR) point cloud quality assessment methods with a fraction of computational cost.

No-reference bitstream-layer models for point cloud quality assessment (PCQA) use the information extracted from a bitstream for real-time and nonintrusive quality monitoring. We propose a no-reference bitstream-layer model for the perceptual quality assessment of video-based point cloud compression (V-PCC) encoded point clouds. First, we study the relationship between the perceptual coding distortion and the texture quantization parameter (TQP) when geometry encoding is lossless. The results indicate that the perceptual coding distortion depends on the texture complexity (TC). Next, we estimate TC using TQP and the texture bitrate per pixel (TBPP), both of which are extracted from the compressed bitstream without resorting to complete decoding. This allows us to build a texture distortion model as a function of TQP and TBPP. By combining this texture distortion model with a geometry distortion model that depends on the geometry quantization parameter (GQP), we obtain an overall no-reference bitstream-layer PCQA model that we call bitstreamPCQ. Experimental results show that the proposed model markedly outperforms existing models in terms of widely used performance criteria, including the Pearson linear correlation coefficient (PLCC), the Spearman rank order correlation coefficient (SRCC) and the root mean square error (RMSE).

3D point clouds have found a wide variety of applications in multimedia processing, remote sensing, and scientific computing. Although most point cloud processing systems are developed to improve viewer experiences, little work has been dedicated to perceptual quality assessment of 3D point clouds. In this work, we build a new 3D point cloud database, namely the Waterloo Point Cloud (WPC) database. In contrast to existing datasets consisting of small-scale and low-quality source content of constrained viewing angles, the WPC database contains 20 high quality, realistic, and omni-directional source point clouds and 740 diversely distorted point clouds. We carry out a subjective quality assessment experiment over the database in a controlled lab environment. Our statistical analysis suggests that existing objective point cloud quality assessment (PCQA) models only achieve limited success in predicting subjective quality ratings. We propose a novel objective PCQA model based on an attention mechanism and a variant of information content-weighted structural similarity, which significantly outperforms existing PCQA models. The database has been made publicly available at https://github.com/qdushl/Waterloo-Point-Cloud-Database.

The emergence of 3D point clouds (3DPCs) is promoting the rapid development of immersive communication, autonomous driving, and so on. Due to the huge data volume, the compression of 3DPCs is becoming more and more attractive. We propose a novel and efficient color attribute compression method for static 3DPCs. First, a 3DPC is partitioned into several sub-point clouds by color distribution analysis. Each sub-point cloud is then decomposed into a lot of 3D blocks by an improved k-d tree-based decomposition algorithm. Afterwards, a novel virtual adaptive sampling-based sparse representation strategy is proposed for each 3D block to remove the redundancy among points, in which the bases of the graph transform (GT) and the discrete cosine transform (DCT) are used as candidates of the complete dictionary. Experimental results over 10 common 3DPCs demonstrate that the proposed method can achieve superior or comparable coding performance when compared with the current state-of-the-art methods.

Recently, 3D point cloud is becoming popular due to its capability to represent the real world for advanced content modality in modern communication systems. In view of its wide applications, especially for immersive communication towards human perception, quality metrics for point clouds are essential. Existing point cloud quality evaluations rely on a full or certain portion of the original point cloud, which severely limits their applications. To overcome this problem, we propose a novel deep learning-based no reference point cloud quality assessment method, namely PQA-Net. Specifically, the PQA-Net consists of a multi-view-based joint feature extraction and fusion (MVFEF) module, a distortion type identification (DTI) module, and a quality vector prediction (QVP) module. The DTI and QVP modules share the feature generated from the MVFEF module. By using the distortion type labels, the DTI and the MVFEF modules are first pre-trained to initialize the network parameters, based on which the whole network is then jointly trained to finally evaluate the point cloud quality. Experimental results on the Waterloo Point Cloud dataset show that PQA-Net achieves better or equivalent performance comparing with the state-of-the-art quality assessment methods. The code of the proposed model will be made publicly available to facilitate reproducible researchhttps://github.com/qdushl/PQA-Net.

In rate-distortion optimization, the encoder settings are determined by maximizing a reconstruction quality measure subject to a constraint on the bitrate. One of the main challenges of this approach is to define a quality measure that can be computed with low computational cost and which correlates well with the perceptual quality. While several quality measures that fulfil these two criteria have been developed for images and videos, no such one exists for point clouds. We address this limitation for the video-based point cloud compression (V-PCC) standard by proposing a linear perceptual quality model whose variables are the V-PCC geometry and color quantization step sizes and whose coefficients can easily be computed from two features extracted from the original point cloud. Subjective quality tests with 400 compressed point clouds show that the proposed model correlates well with the mean opinion score, outperforming state-of-the-art full reference objective measures in terms of Spearman rank-order and Pearson linear correlation coefficient. Moreover, we show that for the same target bitrate, rate-distortion optimization based on the proposed model offers higher perceptual quality than rate-distortion optimization based on exhaustive search with a point-to-point objective quality metric. Our datasets are publicly available at https://github.com/qdushl/Waterloo-Point-Cloud-Database-2.0.

In video-based 3D point cloud compression, the quality of the reconstructed 3D point cloud depends on both the geometry, and color distortions. Finding an optimal allocation of the total bitrate between the geometry coder, and the color coder is a challenging task due to the large number of possible solutions. To solve this bit allocation problem, we first propose analytical distortion, and rate models for the geometry, and color information. Using these models, we formulate the joint bit allocation problem as a constrained convex optimization problem, and solve it with an interior point method. Experimental results show that the rate-distortion performance of the proposed solution is close to that obtained with exhaustive search but at only 0.66$\%$of its time complexity.

The real-world applications of 3D point clouds have been growing rapidly in recent years, but effective approaches and datasets to assess the quality of 3D point clouds are largely lacking. In this work, we construct so far the largest 3D point cloud database with diverse source content and distortion patterns, and carry out a comprehensive subjective user study. We construct 20 high quality, realistic, and omni-directional point clouds of diverse contents. We then apply downsampling, Gaussian noise, and three types of compression algorithms to create 740 distorted point clouds. Based on the database, we carry out a subjective experiment to evaluate the quality of distorted point clouds, and perform a point cloud encoder comparison. Our statistical analysis find that existing point cloud quality assessment models are limited in predicting subjective quality ratings. The database will be made publicly available to facilitate future research.