Preparing subjective evaluations for point cloud quality assessment using 2D or 3D planar displays typically requires rendering uncompressed videos, which demands large volumes of data storage and processing, particularly at high resolutions. This paper describes a subjective quality evaluation of point clouds using a head-mounted display (HMD), which allows the direct 3D content depiction and thus eliminates the need for video rendering. The test dataset included six static point clouds encoded with the MPEG codecs V-PCC and G-PCC, the learning-based codec RS-DLPCC, and the Draco codec. The obtained Mean Opinion Scores (MOS) were compared with two other studies using the same dataset but with different displays, notably a 2D display and a 3D stereoscopic display. The reported results indicate that the three studies are highly correlated, with Pearson and Spearman correlation coefficients above 0.9. Moreover, a multi-way ANOVA shows that changing the display does not have a significant effect on the obtained MOS.

A study on the quality evaluation of point clouds in the presence of coding distortions is presented. For that, four different point cloud coding solutions, notably the standardized MPEG codecs G-PCC and V-PCC, a deep learning-based coding solution RS-DLPCC, and Draco, are compared using a subjective evaluation methodology. Furthermore, several full-reference, reduced-reference and no-reference point cloud quality metrics are evaluated. Two different point cloud normal computation methods were tested for the metrics that rely on them, notably the Cloud Compare quadric fitting method with radius of five, ten, and twenty and Meshlab KNN with K six, ten, and eighteen. To generalize the results, the objective quality metrics were also benchmarked on a public database, with mean opinion scores available. To evaluate the statistical differences between the metrics, the Krasula method was employed. The Point Cloud Quality Metric reveals the best performance and a very good representation of the subjective results, as well as being the metric with the most statistically significant results. It was also revealed that the Cloud Compare quadric fitting method with radius 10 and 20 produced the most reliable normals for the metrics dependent on them. Finally, the study revealed that the most commonly used metrics fail to accurately predict the compression quality when artifacts generated by deep learning methods are present.

In this paper, the quality evaluation of the responses to the Call for Proposals (CfP) of JPEG Pleno Point Cloud Coding is presented. Three responses to the CfP were evaluated together with the state of the art anchor codecs G-PCC and VPCC from MPEG. The JPEG committee selected a set of eight point clouds that were encoded at different pre-established bitrates. For the subjective evaluation of the responses to the CfP, a set of video sequences were created where the reference and distorted decoded point clouds were rotated about their axes side by side. Furthermore, the objective quality metrics PCQM, PSNR D1, PSNR D2, PSNR Y and PSNR YUV were computed, and compared with the subjective evaluation results. This study revealed that the deep learning solutions outperformed G-PCC but were still below the performance of V-PCC regarding color representation. PCQM showed the best performance in predicting the compression quality.

In this paper, a stability analysis of the JPEG Pleno Learning-based Point Cloud Coding Verification Model (VmUC) is performed. The codec is a deep learning-based solution that is able to compress both color and geometry. Three different training sessions were conducted using the default training set and cost function, and six point clouds were encoded/decoded with the resulting operating points for six target distortion/bitrate ratios. The VmUC performance was compared with the MPEG codecs V-PCC and G-PCC, considering three objective metrics, notably PSNR MSE D1, PSNR MSE D2, and PCQM. PSNR MSE D1 was also computed at each training epoch for the six decoded point clouds. It is concluded that the VmUC is able to outperform G-PCC and V-PCC in geometry encoding. However, it is outperformed by V-PCC in terms of color encoding, namely across all three training sessions. Furthermore, it is also shown that the codec does not present a high level of stability, changing its performance considerably with different training sessions.

In this paper, a subjective evaluation of static point clouds encoded with several codecs is described. Unlike other studies, a stereoscopic 3D display was used to visualize the 3D representation. A set of six point clouds were encoded using a set of state of the art point cloud coding solutions, notably the two MPEG codecs V-PCC and G-PCC, a deep learning solution RS-DLPCC that was the response to a call for evidence on point cloud coding of JPEG Pleno, and the popular DRACO codec. The results of this subjective quality evaluation using a 3D representation visualized in a stereoscopic display were compared with a previous subjective study that used the same content visualized in a 2D display. For that, the results of both tests were compared with the Pearson correlation, Spearman rank order correlation, the root mean square error and the outlier ratio. Moreover, the two subjective evaluation results were statistically analysed to seek for any statistical difference. The two subjective evaluations reveal a very high level of similarity.

At its 93rd meeting, JPEG received the responses to the Call for Proposals on JPEG Pleno holography. This activity will lead to the first standardized coding solution for digital holograms. This paper describes the quality evaluation process, existing out of a subjective evaluation study and an objective evaluation based on preselected quality metrics. It also includes a critical evaluation of the challenges encountered when performing quality evaluation of holographic content. Based on the evaluation of the response, the JPEG committee was able to select a very reliable coding solution that depicted a performance well beyond that of the selected anchors.

This paper presents a quality evaluation study of point cloud codecs that have been recently standardised by the MPEG committee. In particular, a subjective experiment to assess their performance in terms of bitrate against visual quality is designed and realized in four independent laboratories. The experimental setup of each laboratory varies; yet, the obtained subjective scores exhibit high inter laboratory correlation, confirming that the adopted assessment protocol is robust to equipment selection and viewing conditions, ensuring reliability and facilitating repeatability. Our study confirms the superior compression performance of the MPEG V-PCC, when compared to MPEG G-PCC, in the case of static contents. Finally, results from a benchmark of the most popular objective quality metrics using the obtained subjective scores as ground truth, reveal that the point2plane with mean square error is the most accurate quality predictor, closely followed by the point2point also using mean square error as distance measure.

Recently stakeholders in the area of multimedia representation and transmission have been looking at plenoptic technologies to improve immersive experience. Among these technologies, point clouds denote a volumetric information representation format with important applications in the entertainment, automotive and geographical mapping industries. There is some consensus that state-of-the-art solutions for efficient storage and communication of point clouds are far from satisfactory. This paper describes a study on point cloud quality evaluation, conducted in the context of JPEG Pleno to help define the test conditions of future compression proposals. A heterogeneous set of static point clouds in terms of number of points, geometric structure and represented scenarios were selected and compressed using octree-pruning and a projection-based method, with three different levels of degradation. The models were comprised of both geometrical and color information and were displayed using point sizes large enough to ensure observation of watertight surfaces. The stimuli under assessment were presented to the observers on 2D displays as animations, after defining suitable camera paths to enable visualization of the models in their entirety and realistic consumption. The experiments were carried out in three different laboratories and the subjective scores were used in a series of correlation studies to benchmark objective quality metrics and assess inter-laboratory consistency.