We introduce Region-Adaptive Learned Hierarchical Encoding (RALHE) for 3D Gaussian Splatting (3DGS) data. While 3DGS has recently become popular for novel view synthesis, the size of trained models limits its deployment in bandwidth-constrained applications such as volumetric media streaming. To address this, we propose a learned hierarchical latent representation that builds upon the principles of “overfitted” learned image compression (e.g., Cool-Chic and C3) to efficiently encode 3DGS attributes. Unlike images, 3DGS data have irregular spatial distributions of Gaussians (geometry) and consist of multiple attributes (signals) defined on the irregular geometry. Our codec is designed to account for these differences between images and 3DGS. Specifically, we leverage the octree structure of the voxelized 3DGS geometry to obtain a hierarchical multi-resolution representation. Our approach overfits latents to each Gaussian attribute under a global rate constraint. These latents are decoded independently through a lightweight decoder network. To estimate the bitrate during training, we employ an autoregressive probability model that leverages octree-derived contexts from the 3D point structure. The multi-resolution latents, decoder, and autoregressive entropy coding networks are jointly optimized for each Gaussian attribute. Experiments on 3DGS models from the Synthetic-NeRF dataset demonstrate that the proposed RALHE compression framework achieves a rendering PSNR gain of up to 2 dB at low bitrates ($\leq 1 \text{MB}$) compared to the baseline 3DGS compression methods.

High-fidelity 3D representations of objects and scenes can be obtained with point clouds, but dealing with their massive data sizes can be difficult. This data is efficiently compressed via MPEG’s Geometry-based Point Cloud Compression (G-PCC), which makes it manageable and useful for real-world applications. One major drawback, though, is that decoding introduces coding artifacts that cause the reconstructed point cloud to appear blocky. In this paper, we present a new approach to attribute learning in point clouds leveraging sparse convolution, that effectively deals with the non-uniformity and sparsity of these data structures.

Subspace Indexing with Interpolation (SIM-I) on Stiefel and Grassmann manifolds is proposed in this work. Given a partition of some original high-dimensional data set, SIM-I is constructed via two steps: in the first step we build linear affinity-aware subspace models based on each partition; in the second step we interpolate between several adjacent linear subspace models constructed in the first step using the “center of mass” calculation on Stiefel and Grassmann manifolds. Through these two steps, SIM-I builds a globally nonlinear and smoothly regularized low-dimensional embedding model of the original data set. Furthermore, given sufficiently many training samples on the data manifold either labelled by some pre-trained learning model such as Deep Neural Networks (DNNs) or provided with original natural labels, we first apply SIM-I on this data set and then perform nearest-neighbor classification on the resulting low-dimensional embedding. This helps us to build a Lightweight Inference Engine (LIE) carrying similar level of feature extraction by the pre-trained learning model. For DNNs, such LIE can be interpreted as some (nonstandard) shallow neural network with a wide first hidden layer. From this perspective, SIM-I provides a way to exchange deep network for wide but shallow ones and may provide some new insights to interpret DNNs.

Point clouds are one of the emerging 3D visual representations of real word and plenty of useful applications has already been demonstrated. However, a huge amount of data associated with it has added challenges in both transmission and storage. This requires an efficient coding solution and brought a great attention among compression community. MPEG and JPEG standardization group has already started developing coding solution and proposed two test-models namely V-PCC, video-based coding solution, for dynamic point cloud and G-PCC, a native geometry-based coding solution, for static and LiDAR point cloud. In G-PCC, octree (lossless) and tri-soup(lossy) for geometry coding, similarly regional adaptive hierarchical transform (RAHT) and lifting-scheme for attributes coding are currently being explored. Lifting-scheme relies on level-of-details(LOD) structure for attributes prediction where LOD is generated with distance based subsampling approach. In this work we proposed a new LOD generation scheme using binary-tree and showed it provides better coding solution for sparse point cloud such as LiDAR. The experimental results demonstrated 12% bitrate reduction for reflectance and 8%, 6% and 7% bitrate reduction for luma, chroma Cb and chroma Cr respectively as well as up to 4 times computational complexity reduction compared to current G-PCC lifting-scheme.

In this paper, based on the Hessian approximation, an incremental pruning method is proposed to compress the deep neural network. The proposed method starts from the idea of using the Hessian to measure the "importance" of each weight in a deep neural network, and it mainly has the following key contributions. First, we propose to use the second moment in Adam optimizer as a measure of the "importance" of each weight to avoid calculating the Hessian matrix. Second, an incremental method is proposed to prune the neural network step by step. The incremental method can adjust the remaining non-zero weights of the whole network after each pruning to help boost the performance of the pruned network. Last but not least, the proposed method applies an automatically-generated global threshold for all the weights among all the layers, which achieves the inter-layer bit allocation automatically. Such a method can improve performance and save the complexity of adjusting the pruning threshold layer by layer. We perform a number of experiments on MNIST and ImageNet using commonly used neural networks such as AlexNet and VGG16 to show the benefits of the proposed algorithm. The experimental results show that the proposed algorithm is able to compress the network significantly with almost no loss of accuracy, which demonstrates the effectiveness of the proposed algorithm.

Point cloud has shown great potential in many different applications including visual communication and auto-navigation systems. However, the immense size of the point cloud has been a bottleneck for its popularity in the consumer level. In this paper, we address the lossless compression of positional data of dynamic point cloud by viewing the point prediction of point cloud as Travelling Salesman Problem (TSP).