An object wave is a suitable intermediate representation format for transmitting holographic application data because it is compatible with various playback devices. However, compressing the phase component of the object wave presents challenges due to its unique signal characteristics, i.e., periodicity and randomness. Hence, object waves significantly differ from ordinary 2D images such as camera-captured and screen pictures. To overcome the challenges, we propose a conversion method for lossless phase image coding using differential pulse-code modulation. Specifically, we calculate phase differences using adjacent pixels in two directions, considering periodic cycles of 2π, and then input them into the existing image coding. This method ensures robustness against randomness by smoothing the phase difference using the average of adjacent pixels. Experimental results show that lossless phase coding with the proposed conversion method achieves coding efficiency improvements of 6.50% over lossless phase coding without the proposed method.

Point cloud quality frequently degrades during various processes, such as scanning, compression, and transmission. Hence, reliable Point Cloud Quality Assessment (PCQA) methods are essential for detecting and mitigating the degradation in 3D applications. This paper proposes an accurate full-reference PCQA method that leverages Multimodal Large Language Models (MLLMs). The proposed method utilizes responses generated by MLLMs to assess point cloud quality. We introduce three innovative PCQA metrics derived from MLLMs: 1) response similarity score, 2) relative quality response score, and 3) absolute quality response score. In addition, we integrate these MLLM-based scores with conventional PCQA metrics using support vector regression to improve accuracy. Experimental results demonstrate that the average Pearson’s Linear Correlation Coefficient (PLCC) and Spearman’s Rank-Order Correlation Coefficient (SROCC) across three datasets improved by 0.046 (from 0.871 to 0.917) and 0.055 (from 0.842 to 0.897), respectively, compared to the state-of-the-art FR-PCQA method.

In real-time applications utilizing point clouds, no-reference point cloud quality assessment (NR-PCQA) methods are essential to improve the accuracy of downstream tasks. For example, in point cloud denoising, NR-PCQA results can be benchmarks for determining the optimal parameters when reference data are unavailable. This paper presents an accurate and fast NR-PCQA method based on graph signal processing. First, we propose new features derived from graph signal variation (GSV) to train a support vector regression model. These features improve the correlation with subjective scores and the robustness against inaccurate graph construction. Second, we present a point selection technique based on graph edge weights that allows us to exclude less relevant points, which results in a precise PCQA. Third, we propose a diagonal scan-line graph (DSLG) construction with a superior tradeoff between accurate and fast graph construction. Our experiments demonstrate improved accuracy and computation time compared with conventional methods with three types of open datasets.

3D mesh models with texture maps play an important role in various 3D applications. Since No-Reference Textured Mesh Quality Assessment (NR-TMQA) methods are essential to identify low-quality mesh models and enhance their quality, this paper presents an accurate NR-TMQA method. Our key contributions are as follows: 1) We propose a novel NR-TMQA scheme that leverages both point cloud and rendered image representations from distorted mesh models. 2) We introduce a novel point cloud representation derived from the faces of a 3D mesh model. It is referred to as a face-based point cloud, which effectively captures face-level features. 3) We present novel graph-based features to estimate mesh quality using the face-based point cloud. Our experimental results demonstrate that Pearson’s linear correlation coefficient and Spearman’s rank-order correlation coefficient in two large-scale TMQA datasets improved by 0.148 (from 0.572 to 0.720) and 0.18 (from 0.525 to 0.705), respectively, compared to the conventional state-of-the-art TMQA method.

Point clouds are utilized in various 3D applications such as cross-reality (XR) and realistic 3D displays. In some applications, e.g., for live streaming using a 3D point cloud, real-time point cloud denoising methods are required to enhance the visual quality. However, conventional high-precision denoising methods cannot be executed in real time for large-scale point clouds owing to the complexity of graph constructions with K nearest neighbors and noise level estimation. This paper proposes a fast graph-based denoising (FGBD) for a large-scale point cloud. First, high-speed graph construction is achieved by scanning a point cloud in various directions and searching adjacent neighborhoods on the scanning lines. Second, we propose a fast noise level estimation method using eigenvalues of the covariance matrix on a graph. Finally, we also propose a new low-cost filter selection method to enhance denoising accuracy to compensate for the degradation caused by the acceleration algorithms. In our experiments, we succeeded in reducing the processing time dramatically while maintaining accuracy relative to conventional denoising methods. Denoising was performed at 30fps, with frames containing approximately 1 million points.

Point clouds in 3D applications frequently experience quality degradation during processing, e.g., scanning and compression. Reliable point cloud quality assessment (PCQA) is important for developing compression algorithms with good bitrate-quality trade-offs and techniques for quality improvement (e.g., denoising). This paper introduces a full-reference (FR) PCQA method utilizing spectral graph wavelets (SGWs). First, we propose novel SGW-based PCQA metrics that compare SGW coefficients of coordinate and color signals between reference and distorted point clouds. Second, we achieve accurate PCQA by integrating several conventional FR metrics and our SGW-based metrics using support vector regression. To our knowledge, this is the first study to introduce SGWs for PCQA. Experimental results demonstrate the proposed PCQA metric is more accurately correlated with subjective quality scores compared to conventional PCQA metrics.

Point clouds are utilized in many 3-D applications such as cross-reality (XR) and realistic 3-D display. They consist of a set of points with 3-D coordinates and associated color signals. These color signals are often perturbed by noise induced by the measurement errors of scanning devices. In this paper, we propose a point cloud denoising method for color signals. Since many conventional methods for point cloud color denoising are based on a low-pass filter in the graph spectral domain, denoising accuracy is affected by the choice of graph. We propose a graph construction method using 3-D patch-based similarity, in which the similarity is calculated with small 3-D patches around the connected points. This is in contrast with conventional graph construction methods for denoising, which are based on point properties such as pairwise point distances and differences in color. Second, we propose a low-pass filtering method where the frequency response is chosen automatically depending on the estimated noise level. Our experimental results show that our proposed method, 3-D patch-based similarity (3DPBS), achieves the best denoising accuracy compared with graph-based state-of-the-art methods.

Point cloud applications suffer from geometric noise caused by measurement errors induced by the point cloud acquisition system. We propose a novel graph construction method, surface-consistent non-negative kernel regression (SC-NNK), that can achieve more accurate denoising of geometry information in combination with spectral graph wavelet transforms (SGWTs). Unlike conventional graph construction methods such as the K-nearest neighbor (KNN), which have been adopted in previous SGWT-based geometry denoising methods, SC-NNK graphs consider geometrical and frequency characteristics to remove redundant edge connections from a KNN graph. In addition, we propose a novel noise level estimation method that achieves improved accuracy by detecting flat surfaces in point clouds, resulting in better wavelet shrinkage thresholds for denoising. Our experimental results show that the proposed method outperforms recent deep-learning-based and graph-based state-of-the-art denoising methods.

Motivated by the success of fractional pixel motion in video coding, we explore the design of motion estimation with fractional-voxel resolution for compression of color attributes of dynamic 3D point clouds. Our proposed block-based fractional-voxel motion estimation scheme takes into account the fundamental differences between point clouds and videos, i.e., the irregularity of the distribution of voxels within a frame and across frames. We show that motion compensation can benefit from the higher resolution reference and more accurate displacements provided by fractional precision. Our proposed scheme significantly outperforms comparable methods that only use integer motion. The proposed scheme can be combined with and add sizeable gains to state-of-the-art systems that use transforms such as Region Adaptive Graph Fourier Transform and Region Adaptive Haar Transform.

We propose a novel graph-based denoising method to correct the quantization error (step noise) arising in the process of generating the visual hull, a commonly used technique to synthesize free-viewpoint video. To reduce this step noise effectively, we propose two new notions of consistency, pixel value consistency and normal vector consistency. The resulting denoising method involves a first step of graph construction using the proposed consistency metrics, followed by graph filtering of the 3D point cloud coordinates. Our experiments show that our approach provides visually and quantitatively better performance than state-of-the-art methods.

We introduce chroma subsampling for 3D point cloud attribute compression by proposing a novel technique to sample points irregularly placed in 3D space. While most current video compression standards use chroma subsampling, these chroma subsampling methods cannot be directly applied to 3D point clouds, given their irregularity and sparsity. In this work, we develop a framework to incorporate chroma subsampling into geometry-based point cloud encoders, such as region adaptive hierarchical transform (RAHT) and region adaptive graph Fourier transform (RAGFT). We propose different sampling patterns on a regular 3D grid to sample the points at different rates. We use a simple graph-based nearest neighbor interpolation technique to reconstruct the full resolution point cloud at the decoder end. Experimental results demonstrate that our proposed method provides significant coding gains with negligible impact on the reconstruction quality. For some sequences, we observe a bitrate reduction of 10-15% under the Bjontegaard metric. More generally, perceptual masking makes it possible to achieve larger bitrate reductions without visible changes in quality.

Many applications that use point clouds, such as 3D immersive telepresence, suffer from geometric quality degradation. This noise may be caused by measurement errors of the capturing device or by the point cloud estimation method. In this paper, we propose a novel graph-based point cloud denoising approach using the spectral graph wavelet transform (SGWT) and graph wavelet shrinkage. Unlike conventional SGWT-based denoising methods, the proposed wavelet shrinkage thresholds are determined based on the normal vector at each point and are thus based on the local geometric structure of the point cloud. This approach avoids excessive wavelet shrinkage, which can lead to the loss of complex geometric structure. Experimental results show that the proposed method achieves the best accuracy as compared with recent deep-learning-based and graph-based state-of-the-art denoising methods.

We propose a novel rendering method for computer-generated holograms (CGHs) which enables low memory usage and low computational complexity. Although the conventional elementary hologram (EH) method realizes smooth motion parallax and rendering for realistic expressions with low complexity, it incurs large memory usage. This is because point light sources (PLSs) must be acquired and stored independently for different perspectives to reconstruct multi-view images. Our method effectively reduces memory usage by integrating PLSs whose 3D coordinates are extremely close to each other. Furthermore, to reduce the complexity of this integration, our method skips some of the PLS acquisition processes under the assumption that the loss of PLSs for the EH does not significantly affect the final reconstructed image quality. Our experimental results show a reduction in memory usage of 93% with a calculation time only 1.2-fold longer than compared with the EH method.

In point cloud compression, exploiting temporal redundancy for inter predictive coding is challenging because of the irregular geometry. This paper proposes an efficient block-based inter-coding scheme for color attribute compression. The scheme includes integer-precision motion estimation and an adaptive graph based in-loop filtering scheme for improved attribute prediction. The proposed block-based motion estimation scheme consists of an initial motion search that exploits geometric and color attributes, followed by a motion refinement that only minimizes color prediction error. To further improve color prediction, we propose a vertex-domain low-pass graph filtering scheme that can adaptively remove noise from predictors computed from motion estimation with different accuracy. Our experiments demonstrate significant coding gain over state-of-the-art coding methods.

Foreground segmentation technologies play an important role in applications such as free-viewpoint video (FVV) and sports video analysis. In this situation, we propose a new method that achieves accurate foreground silhouette extraction method using dynamic object presence probability (DOPP). Our main contributions are as follows. 1) Object presence probability for each pixel is calculated from the object recognition results based on deep learning. After that, background subtraction is implemented by changing the threshold and the update rate of the background model in response to the object presence probability. Parameter tuning of background subtraction is executed by using the object recognition results to improve the silhouette extraction quality. 2) To calculate more accurate silhouette images, parameters of background subtraction are adjusted by monitoring optical flows between consecutive frames. The object presence probability of the current frame is dynamically updated by using the object presence probability of the previous frame with optical flows. In the experiments, we confirmed that the proposed method achieved more accurate silhouette extraction than conventional methods in three sports sequences.

In this paper, we report on a parallel free-viewpoint video synthesis algorithm that can efficiently reconstruct a high-quality 3D scene representation of sports scenes. The proposed method focuses on a scene that is captured by multiple synchronized cameras featuring wide-baselines. The following strategies are introduced to accelerate the production of a free-viewpoint video taking the improvement of visual quality into account: (1) a sparse point cloud is reconstructed using a volumetric visual hull approach, and an exact 3D ROI is found for each object using an efficient connected components labeling algorithm. Next, the reconstruction of a dense point cloud is accelerated by implementing visual hull only in the ROIs; (2) an accurate polyhedral surface mesh is built by estimating the exact intersections between grid cells and the visual hull; (3) the appearance of the reconstructed presentation is reproduced in a view-dependent manner that respectively renders the non-occluded and occluded region with the nearest camera and its neighboring cameras. The production for volleyball and judo sequences demonstrates the effectiveness of our method in terms of both execution time and visual quality.

Free viewpoint technologies that synthesize virtual viewpoint by using multiple actual videos are one of the hottest topics in the video processing field, and would provide immersive experiences for users. To realize this concept, many conventional methods have been proposed. However, these methods require high computational cost to synthesize a virtual viewpoint because they have to calculate huge data to express three-dimensional information, e.g. the shapes of objects, only by using two-dimensional video. This makes them inadequate for end-to-end (from video capture to virtual view rendering) live streaming. To overcome this problem, we propose a simple and fast free-viewpoint synthesis method based on a visual hull, which is a general concept in this field. We calculate the silhouette of an object along planes in virtual space by simple projection from video images to the 3D space, and express the whole shape of the object by integrating the planes. This scheme works very quickly while providing fine quality because it consists of standard functions in general GPU architecture. The experimental results show our method can generate a fine virtual view of an object by using multiple videos in real time.

High Level Synthesis (HLS) has many productivity advantages over traditional RTL design, but routing congestion is difficult to resolve due to the lack of physical information in HLS. In this paper we propose a novel design flow by integrating a HLS tool with physically aware logic synthesis technology. Using this approach, one can discover congestion problems early and trace their sources to specific parts of the input SystemC models. This allows designers to resolve the congestion problems before going to the layout design phase. We applied this flow to a large-scale HLS production design with results showing that this flow can significantly improve not only routing congestion but design area and timing as well.

BACKGROUND: Non-homology based methods such as phylogenetic profiles are effective for predicting functional relationships between proteins with no considerable sequence or structure similarity. Those methods rely heavily on traditional similarity metrics defined on pairs of phylogenetic patterns. Proteins do not exclusively interact in pairs as the final biological function of a protein in the cellular context is often hold by a group of proteins. In order to accurately infer modules of functionally interacting proteins, the consideration of not only direct but also indirect relationships is required. In this paper, we used the Bond Energy Algorithm (BEA) to predict functionally related groups of proteins. With BEA we create clusters of phylogenetic profiles based on the associations of the surrounding elements of the analyzed data using a metric that considers linked relationships among elements in the data set. RESULTS: Using phylogenetic profiles obtained from the Cluster of Orthologous Groups of Proteins (COG) database, we conducted a series of clustering experiments using BEA to predict (upper level) relationships between profiles. We evaluated our results by comparing with COG's functional categories, And even more, with the experimentally determined functional relationships between proteins provided by the DIP and ECOCYC databases. Our results demonstrate that BEA is capable of predicting meaningful modules of functionally related proteins. BEA outperforms traditionally used clustering methods, such as k-means and hierarchical clustering by predicting functional relationships between proteins with higher accuracy. CONCLUSION: This study shows that the linked relationships of phylogenetic profiles obtained by BEA is useful for detecting functional associations between profiles and extending functional modules not found by traditional methods. BEA is capable of detecting relationship among phylogenetic patterns by linking them through a common element shared in a group. Additionally, we discuss how the proposed method may become more powerful if other criteria to classify different levels of protein functional interactions, as gene neighborhood or protein fusion information, is provided.

In this paper, we propose a new method for inferring functional relationships between proteins from their phylogenetic profiles. The evolutionary bond energy algorithm (EBEA) enhances the greedy BEA clustering algorithm with genetic search. We conducted a series of experiments using phylogenetic profiles provided by the Cluster of Orthologous Groups of Proteins (COG) database. Experimental results demonstrate that the proposed method is capable of inferring meaningful clusters of functionally related proteins