We introduce Region-Adaptive Learned Hierarchical Encoding (RALHE) for 3D Gaussian Splatting (3DGS) data. While 3DGS has recently become popular for novel view synthesis, the size of trained models limits its deployment in bandwidth-constrained applications such as volumetric media streaming. To address this, we propose a learned hierarchical latent representation that builds upon the principles of “overfitted” learned image compression (e.g., Cool-Chic and C3) to efficiently encode 3DGS attributes. Unlike images, 3DGS data have irregular spatial distributions of Gaussians (geometry) and consist of multiple attributes (signals) defined on the irregular geometry. Our codec is designed to account for these differences between images and 3DGS. Specifically, we leverage the octree structure of the voxelized 3DGS geometry to obtain a hierarchical multi-resolution representation. Our approach overfits latents to each Gaussian attribute under a global rate constraint. These latents are decoded independently through a lightweight decoder network. To estimate the bitrate during training, we employ an autoregressive probability model that leverages octree-derived contexts from the 3D point structure. The multi-resolution latents, decoder, and autoregressive entropy coding networks are jointly optimized for each Gaussian attribute. Experiments on 3DGS models from the Synthetic-NeRF dataset demonstrate that the proposed RALHE compression framework achieves a rendering PSNR gain of up to 2 dB at low bitrates ($\leq 1 \text{MB}$) compared to the baseline 3DGS compression methods.

Motivated by the success of fractional pixel motion in video coding, we explore the design of motion estimation with fractional-voxel resolution for compression of color attributes of dynamic 3D point clouds. Our proposed block-based fractional-voxel motion estimation scheme takes into account the fundamental differences between point clouds and videos, i.e., the irregularity of the distribution of voxels within a frame and across frames. We show that motion compensation can benefit from the higher resolution reference and more accurate displacements provided by fractional precision. Our proposed scheme significantly outperforms comparable methods that only use integer motion. The proposed scheme can be combined with and add sizeable gains to state-of-the-art systems that use transforms such as Region Adaptive Graph Fourier Transform and Region Adaptive Haar Transform.

In social media analysis, one critical task is detecting a burst of topics or buzz , which is reflected by extremely frequent mentions of certain keywords in a short-time interval. Detecting buzz not only provides useful insights into the information propagation mechanism, but also plays an essential role in preventing malicious rumors. However, buzz modeling is a challenging task because a buzz time-series often exhibits sudden spikes and heavy tails, wherein most existing time-series models fail. In this article, we propose novel buzz modeling approaches that capture the rise and fade temporal patterns via Product Lifecycle (PLC) model, a classical concept in economics. More specifically, we propose to model multiple peaks in buzz time-series with PLC mixture or PLC group mixture and develop a probabilistic graphical model (K-Mixture of Product Lifecycle ( K-MPLC ) to automatically discover inherent lifecycle patterns within a collection of buzzes. Furthermore, we effectively utilize the model parameters of PLC mixture or PLC group mixture for burst prediction. Our experimental results show that our proposed methods significantly outperform existing leading approaches on buzz clustering and buzz-type prediction.

In social media analysis, one critical task is detecting burst of topics or buzz, which is reflected by extremely frequent mentions of certain key words in a short time interval. Detecting buzz not only provides useful insights into the information propagation mechanism, but also plays an essential role in preventing malicious rumors. However, buzz modeling is a challenging task because a buzz time-series usually exhibits sudden spikes and heavy tails, which fails most existing time-series models. To deal with buzz time-series sequences, we propose a novel time-series modeling approach which captures the rise and fade temporal patterns via Product Life Cycle (PLC) models, a classical concept in economics. More specifically, we propose a mixture of PLC models to capture the multiple peaks in buzz time-series and furthermore develop a probabilistic graphical model (K-MPLC) to automatically discover inherent life cycle patterns within a collection of buzzes. Our experiment results show that our proposed method significantly outperforms existing state-of-the-art approaches on buzzes clustering.

We consider the problem of offline, pool-based active semi-supervised learning on graphs. This problem is important when the labeled data is scarce and expensive whereas unlabeled data is easily available. The data points are represented by the vertices of an undirected graph with the similarity between them captured by the edge weights. Given a target number of nodes to label, the goal is to choose those nodes that are most informative and then predict the unknown labels. We propose a novel framework for this problem based on our recent results on sampling theory for graph signals. A graph signal is a real-valued function defined on each node of the graph. A notion of frequency for such signals can be defined using the spectrum of the graph Laplacian matrix. The sampling theory for graph signals aims to extend the traditional Nyquist-Shannon sampling theory by allowing us to identify the class of graph signals that can be reconstructed from their values on a subset of vertices. This approach allows us to define a criterion for active learning based on sampling set selection which aims at maximizing the frequency of the signals that can be reconstructed from their samples on the set. Experiments show the effectiveness of our method.

We present a new graph-based transform for video signals using wavelet lifting. Graphs are created to capture spatial and temporal correlations in video sequences. Our new transforms allow spatial and temporal correlation to be jointly exploited, in contrast to existing techniques, such as motion compensated temporal filtering, which can be seen as "separable" transforms, since spatial and temporal filtering are performed separately. We design efficient ways to form the graphs and to design the prediction and update filters for different levels of the lifting transform as a function of expected degree of correlation between pixels. Our initial results are promising, with improvements in performance as compared to existing methods in terms of PSNR as a function of the percentage of retained coefficients of the transform.

Remote visualization of an arbitrary 2-D planar "cut" from a large volumetric dataset with random access has both gained importance and posed significant challenges over the past few years in industrial and medical applications. In this paper, a prediction model is presented that relates transmission efficiency to voxel coverage statistics for a fast random 2-D image retrieval system. This model can be for parameter selection and also provides insights that lead us to propose a new 3D rectangular tiling scheme, which achieves an additional 10% - 30% reduction in average transmission rate as compared to our previously proposed technique, e.g.,a nearly 30%/45% reduction in the average transmission rate at the cost of a factor of ten/fifteen in storage overhead compared to traditional cubic tiling. Furthermore, this approach leads to improved random access, with less storage and run-time memory required at the client.

Volume visualization with random data access poses significant challenges. While tiling techniques lead to simple implementations, they are not well suited for cases where the goal is to access arbitrarily located subdimensional datasets (e.g., being able to display an arbitrary 2D planar ldquocutrdquo from a 3D volume). Significant effort has been devoted to volumetric data compression, with most techniques proposing to tile volumes into cuboid subvolumes to enable random access. In this paper we show that, in cases where subdimensional datasets are accessed, this leads to significant transmission inefficiency. As an alternative, we propose novel server-client based data representation and retrieval methods which can be used for fast random access of oblique plane from 3D volume datasets. In this paper, 3D experiments are shown but the approach may be extended to higher dimensional datasets. We use multiple redundant tilings of the 3D object, where each tiling has a different orientation.We discuss the 3D rectangular tiling scheme and two main algorithm components of such 3D system, namely, (i) a search algorithm to determine which tiles should be retrieved for a given query and (ii) a mapping algorithm to enable efficient encoding without interpolation of rotated tiles. In exchange for increased server storage, we demonstrate that significant reductions in average transmission rate can be achieved relative to conventional cubic tiling techniques, e.g., nearly 40% reduction in average transmission rate for less than a factor of twenty overhead in storage before compression. Note that, as shown in our earlier work on the 2D case, the storage overhead will be lower after compression (e.g., in 2D the relative increase in storage in the compressed domain was at least a factor of two lower than in the uncompressed domain).

Hyperspectral images have correlation at the level of pixels; moreover, images from neighboring frequency bands are also closely correlated. In this paper, we propose to use distributed source coding to exploit this correlation with an eye to a more efficient hardware implementation. Slepian-Wolf and Wyner-Ziv based correlated coding theorems have quantified how much additional rate reduction can be obtained. In order to better exploit these correlations, we first propose a prediction model to align images. This model is based on linear prediction techniques and it is simple and shown to be effective for hyperspectral images. We then propose a coding scheme to exploit these correlations. A set-partitioning approach is used on wavelet transformed data to extract bitplanes. Under our correlation model, bitplanes from neighboring bands are correlated and we then use a low-density parity-check based Slepian-Wolf code to exploit this bitplane level correlation. This scheme is appealing for hardware implementation as it is easy to parallelize and it has modest memory requirements. As for coding performance, our preliminary results for high correlation spectral bands from the NASA AVIRIS dataset show, at medium to high reconstructed qualities, gains of about a factor of 3 in compression efficiency as compared to encoding the spectral bands independently using SPIHT.

In optimal product vector quantization (VQ) sub-vectors within a vector are encoded separately. Optimal product VQ (PVQ) aims at maximizing the rate-distortion (RD) performance. We consider scenarios where PVQ is used to approximate the labeling obtained from an existing higher dimension quantizer or classifier. We present an efficient design technique under the labeling constraints and we show that performance is significantly improved if these are taken into account. We present two examples where this technique can be used. First we consider a PVQ designed to approximate a higher dimension classifier. In this case we show that with a small penalty in distortion (e.g., 0.04 dB loss) we can reduce significantly the misclassification (e.g., 48% relative reduction, 4.6% absolute reduction) with respect to a standard PVQ design. In our second example we show how hierarchical VQ (HVQ) can be used as a preprocessing stage for a standard unstructured VQ such that the HVQ stage enables a significant reduction of the codeword candidates to be searched in the VQ stage. Here again we show how HVQ designed to optimize the labeling enables a further reduction in complexity as the HVQ partition is designed to approximate the standard VQ partition.

We study the construction of structured regular quantizers for overcomplete expansions in R/sup N/. Our goal is to design structured quantizers allowing simple reconstruction algorithms with low (memory and computational) complexity and having good performance in terms of accuracy. Most related work to date in quantized redundant expansions has assumed that uniform scalar quantization with the same stepsize was used on the redundant expansion and then has dealt with more complex methods to improve the reconstruction. Instead, we consider the design of scalar quantizers with different stepsizes for each coefficient of an overcomplete expansion in such a way as to produce an equivalent vector quantizer with periodic structure. The periodicity makes it possible to achieve good accuracy using simple reconstruction algorithms from the quantized coefficients of the overcomplete expansion.

In applications where compression has to be performed under varying complexity constraints (e.g., with hardware having to operate in reduced power mode) it is beneficial to design compression algorithms that allow some degree of complexity scalability. In this paper we explore complexity scalability for transform coding algorithms. We show that a variable complexity algorithm (VCA), which uses energy thresholds to determine the number of coefficients to be computed for each input, is preferable to other alternatives such as a pruned transform, where the same number of coefficients is computed for the whole image. We show that the benefits include not only a higher degree of scalability, but also increased compression performance, as we take advantage of the energy classification that is needed for VCA operation and design quantizers that match each class. We provide expressions for the average complexity as well as rate/distortion relations for a generic N-point VCA transform. For a two point case, we present closed-form relations describing the variance changes in two classes. In addition, rate-distortion-complexity relations are also empirically obtained. We apply VCA to eight-point KLT and 8/spl times/8 DCT in the JPEG framework and experiments show that the VCA approach is superior in rate/distortion performance at low rates compared to the standard transform coding techniques.

We consider the problem of searching for the best match for an input among a set of vectors, according to some predetermined metric. Examples of this problem include the search for the best match for an input in a VQ encoder and the search for a motion vector in motion estimation-based video coding. We propose an approach that computes a partial distance metric and uses prior probabilistic knowledge of the reliability of the estimate to decide on whether to stop the distance computation. This is achieved with a simple hypothesis testing and the result, an extension of the partial distance technique of Bei and Gray (1985) provides additional computation savings at the cost of a (controllable) loss in matching performance.

In this work we propose a novel algorithm for wavelet based image compression with very low memory requirements. The wavelet transform is performed progressively and we only require that a reduced number of lines from the original image be stored at any given time. The result of the wavelet transform is the same as if we were operating on the whole image, the only difference being that the coefficients of different subbands are generated in an interleaved fashion. We begin encoding the (interleaved) wavelet coefficients as soon as they become available. We classify each new coefficient in one of several classes, each corresponding to a different probability model, with the models being adapted on the fly for each image. Our scheme is fully backward adaptive and it relies only on coefficients that have already been transmitted. Our experiments demonstrate that our coder is still very competitive with respect to similar state-of-the-art coders. It is noted that schemes based on zero trees or bit plane encoding basically require the whole image to be transformed (or else have to be implemented using tiling). The features of the algorithm make it well suited for a low memory mode coding within the emerging JPEG2000 standard.

We present an adaptive image coding algorithm based on novel backward-adaptive quantization/classification techniques. We use a simple uniform scalar quantizer to quantize the image subbands. Our algorithm puts the coefficient into one of several classes depending on the values of neighboring previously quantized coefficients. These previously quantized coefficients form contexts which are used to characterize the subband data. To each context type corresponds a different probability model and thus each subband coefficient is compressed with an arithmetic coder having the appropriate model depending on that coefficient's neighborhood. We show how the context selection can be driven by rate-distortion criteria, by choosing the contexts in a way that the total distortion for a given bit rate is minimized. Moreover the probability models for each context are initialized/updated in a very efficient way so that practically no overhead information has to be sent to the decoder. Our results are comparable or in some cases better than the recent state of the art, with our algorithm being simpler than most of the published algorithms of comparable performance.

We present a new Lagrangian-based iterative technique for rate-distortion optimization under multiple rate constraints. We show how for sets of "linear" constraints this technique can be proven to be optimal up to a convex hull approximation. As an application we consider the problem of optimal buffer-constrained bit allocation. Our technique can be used to find an excellent approximation to the solutions achieved using dynamic programming. In cases where the buffer size is relatively large our approach shows a significant reduction in complexity as compared to dynamic programming.