The evaluation of perceptual quality in 3D mesh compression, particularly for Video-based Dynamic Mesh Coding (V-DMC), is challenged by the scarcity of subject-rated datasets and the high computational cost of full-mesh decoding and the sophisticated visual feature extraction steps. To bridge this gap, we first introduce a novel V-DMC distortion dataset, comprising 16 high-quality original meshes and 400 compressed, textureless variants. We conducted a subjective quality assessment study with 30 participants using the Double Stimulus Impairment Scale (DSIS) method to collect reliable Mean Opinion Scores (MOS). We then propose streamMQ, the first-of-its-kind no-reference, bitstream-layer model for perceptual quality assessment of V-DMC compressed meshes. By extracting key geometric features such as quantization parameters and triangle count directly from the compressed bitstream, streamMQ predicts perceptual quality without full decoding. Experimental evaluation and comparison with state-of-the-art methods demonstrate that streamMQ achieves highly competitive quality assessment performance at tiny fractions of computational and storage costs, facilitating real-time and low-storage application environments. The dataset and source code will be made publicly available at https://github.com/HFL01/QDU-GDM.

The visual quality of point clouds is critical for perception-centric immersive media. Point Cloud Quality Assessment (PCQA) is crucial for reducing costs associated with human evaluation, optimizing compression pipeline and enhancing human visual perception. However, real-valued PCQA methods often struggle to capture the coupled geometric and perceptual cues that govern quality. Com-PCQA, a novel no-reference PCQA framework leveraging complex-valued feature learning, is proposed. First, a Hilbert dual-stream module transforms multi-modal inputs of point clouds and images into analytic signals in the complex domain, enabling joint modeling of global structure and local texture with efficient tensor operations. Second, a complex amplitude-phase attention (CAPA) module explicitly decomposes and fuses amplitude features that describe geometric structure and phase features that capture fine-grained details, and it can be seamlessly integrated into other PCQA frameworks to enhance performance. Third, an adversarial joint scoring module integrates adversarial training with collaborative learning to calibrate multi-modal, multi-scale representations and enhance robustness. Extensive experiments on three public databases show that Com-PCQA achieves state-of-the-art correlations with subjective scores and consistently outperforms recent PCQA methods, demonstrating its effectiveness and robustness. The code will be available at https://openi.pcl.ac.cn/OpenPointCloud and https://github.com/LareinaSu/Com-PCQA.

No-reference bitstream-layer point cloud quality assessment (PCQA) can be deployed without full decoding at any network node to achieve real-time quality monitoring. In this work, we develop the first PCQA model dedicated to Trisoup-Lifting encoded 3D point clouds by analyzing bitstreams without full decoding. Specifically, we investigate the relationship among texture bitrate per point (TBPP), texture complexity (TC) and texture quantization parameter (TQP) while geometry encoding is lossless. Subsequently, we estimate TC by utilizing TQP and TBPP. Then, we establish a texture distortion evaluation model based on TC, TBPP and TQP. Ultimately, by integrating this texture distortion model with a geometry attenuation factor, a function of trisoupNodeSizeLog2 (tNSL), we acquire a comprehensive NR bitstream-layer PCQA model named streamPCQ-TL. In addition, this work establishes a database named WPC6.0, the first PCQA database dedicated to Trisoup-Lifting encoding mode, encompassing 400 distorted point clouds with 4 geometry multiplied by 5 texture distortion levels. Experiment results on M-PCCD, ICIP2020 and the proposed WPC6.0 database suggest that the proposed streamPCQ-TL model exhibits robust and notable performance in contrast to existing advanced PCQA metrics, particularly in terms of computational cost.

Due to the non-uniform perception of human vision, structural or color changes in salient regions play a dominant role in point cloud quality assessment (PCQA). In this paper, we propose a perception-weighted multi-view PCQA method based on saliency-guided coverage analysis (PW-SCQA), which dynamically quantifies the contribution of different viewpoints on perceptual quality through saliency. First, multi-view projection images are generated based on a polyhedral projection mechanism and multi-scale features are extracted for constructing a 2D saliency map. Then, the 2D to 3D saliency propagation model is used to refine the point-level saliency weights and achieve point cloud saliency visualization. Subsequently, a perception-driven viewpoint optimization mechanism and a novel viewpoint saliency region coverage (SRC) index are innovatively introduced, in which the viewpoint evaluation weights are dynamically adjusted by calculating the SRC of high, medium, and low saliency under the candidate viewpoints. Finally, the multi-view information content weighting image quality assessment method is combined to predict the overall point cloud quality. PW-SCQA outperforms several state-of-the-art methods on three different PCQA datasets.

With the rapid development of immersive multimedia technology, the growing demand for high-quality visual experiences has driven the emergence of point cloud quality assessment (PCQA). While current deep learning-based PCQA models have achieved breakthroughs in performance, problems such as high computational complexity and limited model generalization ability still need to be solved. In this study, focusing on compression distortion, we analyzed and verified that the compression quantization parameter (QP) can be used as a key feature for predicting perceptual quality. Based on this, a novel no-reference point cloud perceptual quality assessment metric, DQP-PCQA, is proposed. Unlike existing PCQA models that only use mean opinion score (MOS) as a supervisory label, this study proposes a multi-objective constrained optimization scheme that adds geometric quantization parameter (GQP) and texture quantization parameter (TQP) as auxiliary supervisory labels to help the model can learn robust perceptual features that take into account both subjective quality and objective distortion. We conducted comparative experiments with other advanced PCQA models on several mainstream PCQA datasets. The results show that the DQP-PCQA model achieves fast convergence speed, excellent and stable performance, low complexity and strong generalization. Further migration experiments show that after applying our proposed method to other advanced PCQA models, the performance of the improved model is further improved. Our discovery provides new insight for PCQA research. To facilitate future reproducible research, the source code will be publicly released at https://github.com/Dds46/DQP-PCQA.

Point cloud compression significantly reduces data volume but sacrifices reconstruction quality, highlighting the need for advanced quality enhancement techniques. Most existing approaches focus primarily on point-to-point fidelity, often neglecting the importance of perceptual quality as interpreted by the human visual system. To address this issue, we propose a generative adversarial network for point cloud quality enhancement (PCE-GAN), grounded in optimal transport theory, with the goal of simultaneously optimizing both data fidelity and perceptual quality. The generator consists of a local feature extraction (LFE) unit, a global spatial correlation (GSC) unit and a feature squeeze unit. The LFE unit uses dynamic graph construction and a graph attention mechanism to efficiently extract local features, placing greater emphasis on points with severe distortion. The GSC unit uses the geometry information of neighboring patches to construct an extended local neighborhood and introduces a transformer-style structure to capture long-range global correlations. The discriminator computes the deviation between the probability distributions of the enhanced point cloud and the original point cloud, guiding the generator to achieve high quality reconstruction. Experimental results show that the proposed method achieves state-of-the-art performance. Specifically, when applying PCE-GAN to the latest geometry-based point cloud compression (G-PCC) test model, it achieves an average BD-rate of -19.2% compared with the PredLift coding configuration and -18.3% compared with the RAHT coding configuration. Subjective comparisons show a significant improvement in texture clarity and color transitions, revealing finer details and more natural color gradients.

The scope of point cloud (PC) applications is expanding. We propose a no-reference bitstream-layer quality assessment model that eliminates the need for full decoding of the PC, providing quality evaluation scores during the V-PCC decoding process. Specifically, we illustrate the relationship between content diversity (CD) and perceptual coding distortion in lossless geometric coding. Subsequently, we model attribute distortion by predicting CD using transform energy (TE) and texture quantization parameter (TQP). By combining the geometric distortion model with geometry quantization parameters (GQP) and the attribute distortion model, we derive comprehensive quality prediction results. Our experimental results on four PC databases (WPC2.0, M-PCCD, VSENSE VVDB and VSENSE VVDB2) show that the proposed energy-adaptive bitstream-layer model (EABL) delivers competitive quality prediction performance in comparison with existing full-reference, reduced-reference and no-reference PC quality assessment models that require full decoding, and meanwhile exhibits large speed advantage. The source code will be made publicly available for repeatability research at https://github.com/arthas-sws/EABL_model.

No-reference point cloud quality assessment (PCQA) based on bitstreams uses information extracted from the bitstream for quality monitoring at network nodes. We develop a no-reference PCQA model based on bitstreams for the perceived quality assessment of Octree-Lifting coded point clouds. At first, our research explores the essential correlation between subjective visual quality degradation and the texture quantization parameter (TQP) when using lossless geometric coding. Then, we enhance the proposed model by incorporating texture complexity (TC) while taking into account the dependence of perceptual coding distortion on the texture characteristics of a point cloud. We estimate TC by utilizing TQP and calculating the average standard deviation of the Y-component of the attribute value ($Y\_ {std}$Y_std), both of which are extracted from the bitstream. Then, a texture distortion assessment model is constructed based on TQP and $Y\_ {std}$Y_std. The integration of the texture distortion model with the position quantization scale (PQS) results in the derivation of an overall no-reference bitstream-based PCQA model, named streamPCQ-OL. The findings from the conducted experiments highlight a significant superiority of the proposed model over existing approaches in terms of performance.

Point cloud perceptual quality assessment plays a critical role in many applications, including compression and communication. We propose PKT-PCQA, a point-based no-reference point cloud quality assessment deep learning network that emulates the human visual system by using progressive knowledge transfer to convert coarse-grained quality classification knowledge into a fine-grained quality prediction task. PKT-PCQA exploits local and global features, as well as an attention mechanism based on spatial and channel attention modules. Experiments on three large and independent point cloud assessment datasets show that PKT-PCQA outperforms existing no-reference and reduced-reference point cloud quality assessment methods and achieves better or similar performance compared to several State-of-the-Art full-reference methods.

Video-based point cloud compression (V-PCC) is a state-of-the-art moving picture experts group (MPEG) standard for point cloud compression. V-PCC can be used to compress both static and dynamic point clouds in a lossless, near lossless, or lossy way. Many objective quality metrics have been proposed for distorted point clouds. Most of these metrics are full-reference metrics that require both the original point cloud and the distorted one. However, in some real-time applications, the original point cloud is not available, and no-reference or reduced-reference quality metrics are needed. Three main challenges in the design of a reduced-reference quality metric are how to build a set of features that characterize the visual quality of the distorted point cloud, how to select the most effective features from this set, and how to map the selected features to a perceptual quality score. We address the first challenge by proposing a comprehensive set of features consisting of compression, geometry, normal, curvature, and luminance features. To deal with the second challenge, we use the least absolute shrinkage and selection operator (LASSO) method, which is a variable selection method for regression problems. Finally, we map the selected features to the mean opinion score in a nonlinear space. Although we have used only 19 features in our current implementation, our metric is flexible enough to allow any number of features, including future more effective ones. Experimental results on the Waterloo point cloud dataset version 2 (WPC2.0) and the MPEG point cloud compression dataset (M-PCCD) show that our method, namely PCQAML, outperforms state-of-the-art full-reference and reduced-reference quality metrics in terms of Pearson linear correlation coefficient, Spearman rank order correlation coefficient, Kendall's rank-order correlation coefficient, and root mean squared error.

Point clouds are rapidly gaining popularity in many practical applications, and point cloud quality assessment (PCQA) is an important research topic that helps us measure and improve the visual experience in applications using point clouds. Research on full-reference (FR) PCQAs has recently made impressive progress, and research on no-reference (NR) PCQAs has also gradually increased. However, the performance of the prior NR PCQA methods still suffers from weak generalization ability and lower accuracy than the FR metrics in general. In this work, we propose a two-stage sampling method that can reasonably represent a whole point cloud, making it possible to efficiently calculate the point cloud quality. For quality prediction, we designed a twin-attention-based transformer PCQA model (3DTA), which uses the data of the two-stage sampling method as input and directly outputs the predicted quality score. Our model is accurate and widely applicable, and it has a simple and flexible structure. Experimental results show that in most cases, the proposed 3DTA model substantially outperforms the benchmark NR methods. The accuracy of the proposed method is competitive even against that of the FR method, which makes 3DTA a strong candidate for the PCQA task, regardless of the reference availability. The code of the proposed model is publicly available athttps://github.com/philox12358/3DTA-PCQA.

With the rapid development of 3D vision applications based on point clouds, point cloud quality assessment (PCQA) is becoming an important research topic.However, the prior PCQA methods ignore the effect of local quality variance across different areas of the point cloud.To take an advantage of the quality distribution imbalance, we propose a no-reference point cloud quality assessment (NR-PCQA) method with local area correlation analysis capability, denoted as COPP-Net.More specifically, we split a point cloud into patches, generate texture and structure features for each patch, and fuse them into patch features to predict patch quality.Then, we gather the features of all the patches of a point cloud for correlation analysis, to obtain the correlation weights.Finally, the predicted qualities and correlation weights for all the patches are used to derive the final quality score.Experimental results show that our method outperforms the state-of-the-art benchmark NR-PCQA methods.The source code for the proposed COPP-Net can be found at https://github.com/philox12358/COPP-Net.

With the increasing demand of compressing and streaming 3D point clouds under constrained bandwidth, it has become ever more important to accurately and efficiently determine the quality of compressed point clouds, so as to assess and optimize the quality-of-experience (QoE) of end users. Here we make one of the first attempts developing a bitstream-based no-reference (NR) model for perceptual quality assessment of point clouds without resorting to full decoding of the compressed data stream. Specifically, we first establish a relationship between texture complexity and the bitrate and texture quantization parameters based on an empirical rate-distortion model. We then construct a texture distortion assessment model upon texture complexity and quantization parameters. By combining this texture distortion model with a geometric distortion model derived from Trisoup geometry encoding parameters, we obtain an overall bitstream-based NR point cloud quality model named streamPCQ. Experimental results show that the proposed streamPCQ model demonstrates highly competitive performance when compared with existing classic full-reference (FR) and reduced-reference (RR) point cloud quality assessment methods with a fraction of computational cost.

No-reference bitstream-layer models for point cloud quality assessment (PCQA) use the information extracted from a bitstream for real-time and nonintrusive quality monitoring. We propose a no-reference bitstream-layer model for the perceptual quality assessment of video-based point cloud compression (V-PCC) encoded point clouds. First, we study the relationship between the perceptual coding distortion and the texture quantization parameter (TQP) when geometry encoding is lossless. The results indicate that the perceptual coding distortion depends on the texture complexity (TC). Next, we estimate TC using TQP and the texture bitrate per pixel (TBPP), both of which are extracted from the compressed bitstream without resorting to complete decoding. This allows us to build a texture distortion model as a function of TQP and TBPP. By combining this texture distortion model with a geometry distortion model that depends on the geometry quantization parameter (GQP), we obtain an overall no-reference bitstream-layer PCQA model that we call bitstreamPCQ. Experimental results show that the proposed model markedly outperforms existing models in terms of widely used performance criteria, including the Pearson linear correlation coefficient (PLCC), the Spearman rank order correlation coefficient (SRCC) and the root mean square error (RMSE).

3D point clouds have found a wide variety of applications in multimedia processing, remote sensing, and scientific computing. Although most point cloud processing systems are developed to improve viewer experiences, little work has been dedicated to perceptual quality assessment of 3D point clouds. In this work, we build a new 3D point cloud database, namely the Waterloo Point Cloud (WPC) database. In contrast to existing datasets consisting of small-scale and low-quality source content of constrained viewing angles, the WPC database contains 20 high quality, realistic, and omni-directional source point clouds and 740 diversely distorted point clouds. We carry out a subjective quality assessment experiment over the database in a controlled lab environment. Our statistical analysis suggests that existing objective point cloud quality assessment (PCQA) models only achieve limited success in predicting subjective quality ratings. We propose a novel objective PCQA model based on an attention mechanism and a variant of information content-weighted structural similarity, which significantly outperforms existing PCQA models. The database has been made publicly available at https://github.com/qdushl/Waterloo-Point-Cloud-Database.

Hierarchical structures of labels usually exist in large-scale classification tasks, where labels can be organized into a tree-shaped structure. The nodes near the root stand for coarser labels, while the nodes close to leaves mean the finer labels. We label unseen samples from the root node to a leaf node, and obtain multigranularity predictions in the hierarchical classification. Sometimes, we cannot obtain a leaf decision due to uncertainty or incomplete information. In this case, we should stop at an internal node, rather than going ahead rashly. However, most existing hierarchical classification models aim at maximizing the percentage of correct predictions, and do not take the risk of misclassifications into account. Such risk is critically important in some real-world applications, and can be measured by the distance between the ground truth and the predicted classes in the class hierarchy. In this work, we utilize the semantic hierarchy to define the classification risk and design an optimization technique to reduce such risk. By defining the conservative risk and the precipitant risk as two competing risk factors, we construct the balanced conservative/precipitant semantic (BCPS) risk matrix across all nodes in the semantic hierarchy with user-defined weights to adjust the tradeoff between two kinds of risks. We then model the classification process on the semantic hierarchy as a sequential decision-making task. We design an algorithm to derive the risk-minimized predictions. There are two modules in this model: 1) multitask hierarchical learning and 2) deep reinforce multigranularity learning. The first one learns classification confidence scores of multiple levels. These scores are then fed into deep reinforced multigranularity learning for obtaining a global risk-minimized prediction with flexible granularity. Experimental results show that the proposed model outperforms state-of-the-art methods on seven large-scale classification datasets with the semantic tree.

Abstract In stereoscopic image quality assessment, human visual system has been universally taken into account to detect perceptual characteristics. A novel full‐reference stereoscopic image assessment metric by considering both monocular and binocular visual features of human visual system is proposed. In particular, a new region segmentation algorithm is firstly proposed to divide 3D images into occluded and non‐occluded regions. The just noticeable difference model is employed on the occluded regions to formulate the monocular vision, while the binocular just noticeable difference model is applied to the non‐occluded regions to reveal the binocular vision of the human visual system. In the proposed region segmentation, disparity information and Euclidean distance between stereo pairs are both adopted to solve the unstable segmentation problem of traditional methods. A new pooling strategy based on global edge features is then presented to aggregate the just noticeable difference and binocular just noticeable difference evaluation maps. In addition, some local image features as supplementary of just noticeable difference to describe visual characteristics of the human visual system are also extracted. Finally, an overall quality score is calculated based on the above‐mentioned features to measure the visual quality of distorted stereo pairs. Experimental results show that the proposed metric achieves high consistency with the human visual system, and outperforms state‐of‐the‐art algorithms on stereoscopic image quality assessment.

Recently, 3D point cloud is becoming popular due to its capability to represent the real world for advanced content modality in modern communication systems. In view of its wide applications, especially for immersive communication towards human perception, quality metrics for point clouds are essential. Existing point cloud quality evaluations rely on a full or certain portion of the original point cloud, which severely limits their applications. To overcome this problem, we propose a novel deep learning-based no reference point cloud quality assessment method, namely PQA-Net. Specifically, the PQA-Net consists of a multi-view-based joint feature extraction and fusion (MVFEF) module, a distortion type identification (DTI) module, and a quality vector prediction (QVP) module. The DTI and QVP modules share the feature generated from the MVFEF module. By using the distortion type labels, the DTI and the MVFEF modules are first pre-trained to initialize the network parameters, based on which the whole network is then jointly trained to finally evaluate the point cloud quality. Experimental results on the Waterloo Point Cloud dataset show that PQA-Net achieves better or equivalent performance comparing with the state-of-the-art quality assessment methods. The code of the proposed model will be made publicly available to facilitate reproducible researchhttps://github.com/qdushl/PQA-Net.

In rate-distortion optimization, the encoder settings are determined by maximizing a reconstruction quality measure subject to a constraint on the bitrate. One of the main challenges of this approach is to define a quality measure that can be computed with low computational cost and which correlates well with the perceptual quality. While several quality measures that fulfil these two criteria have been developed for images and videos, no such one exists for point clouds. We address this limitation for the video-based point cloud compression (V-PCC) standard by proposing a linear perceptual quality model whose variables are the V-PCC geometry and color quantization step sizes and whose coefficients can easily be computed from two features extracted from the original point cloud. Subjective quality tests with 400 compressed point clouds show that the proposed model correlates well with the mean opinion score, outperforming state-of-the-art full reference objective measures in terms of Spearman rank-order and Pearson linear correlation coefficient. Moreover, we show that for the same target bitrate, rate-distortion optimization based on the proposed model offers higher perceptual quality than rate-distortion optimization based on exhaustive search with a point-to-point objective quality metric. Our datasets are publicly available at https://github.com/qdushl/Waterloo-Point-Cloud-Database-2.0.

In video-based 3D point cloud compression, the quality of the reconstructed 3D point cloud depends on both the geometry, and color distortions. Finding an optimal allocation of the total bitrate between the geometry coder, and the color coder is a challenging task due to the large number of possible solutions. To solve this bit allocation problem, we first propose analytical distortion, and rate models for the geometry, and color information. Using these models, we formulate the joint bit allocation problem as a constrained convex optimization problem, and solve it with an interior point method. Experimental results show that the rate-distortion performance of the proposed solution is close to that obtained with exhaustive search but at only 0.66$\%$of its time complexity.

The real-world applications of 3D point clouds have been growing rapidly in recent years, but effective approaches and datasets to assess the quality of 3D point clouds are largely lacking. In this work, we construct so far the largest 3D point cloud database with diverse source content and distortion patterns, and carry out a comprehensive subjective user study. We construct 20 high quality, realistic, and omni-directional point clouds of diverse contents. We then apply downsampling, Gaussian noise, and three types of compression algorithms to create 740 distorted point clouds. Based on the database, we carry out a subjective experiment to evaluate the quality of distorted point clouds, and perform a point cloud encoder comparison. Our statistical analysis find that existing point cloud quality assessment models are limited in predicting subjective quality ratings. The database will be made publicly available to facilitate future research.

A novel bitrate model with low complexity is proposed for perceived compression distortion assessment of mobile video with low resolution, which is extremely useful in intermediate network nodes for quality monitoring. Without fully decoding, parameters are extracted by bitstream analysing, such as bitrate, frame type, quantisation parameter, DCT coefficient, motion vector. Bitrate is regarded as an essential parameter meanwhile the bitrate–MOS curve is determined by video content. Respectively, spatial factor is estimated using quantisation parameter and DCT coefficient and temporal factor is estimated using motion vector. Apart from bitrate, the spatial and temporal factors, which reflect the characteristic of video content, are considered in the proposed model to obtain a more accurate evaluation. Experimental results show that the overall performance of proposed model significantly outperforms that of the other five bitrate models in terms of widely used performance criteria, including the Pearson correlation coefficient (PCC), the Spearman rank‐order correlation coefficient (SROCC), the root‐mean‐squared error (RMSE) and the outlier ratio (OR).