We present a novel spherical image compression approach based on Spherical Transformer Block(STB) that takes advantage of HEALPix's hierarchical structure, as illustrated in Figure 1. More specifically, we exploit the hierarchical properties of HEALPix to partition the sphere into several smaller windows and apply multi-head attention within each window, thereby alleviating the computational burden of the spherical Transformer. However, as there is no communication between neighboring windows, we incorporate spherical convolution to aggregate information from adjacent windows. In addition, we propose an inverted residual bottleneck module for feature embedding and a feedforward neural network. Experimental results show that our method outperforms the OSLO [1] approach in terms of performance, while also achieving faster codec times.

End-to-end learned image compression exploits the expressive power of nonlinear transform modules to de-correlate the spatial redundancies of image contents. Due to its long-range attention scheme, transformer-based transforms can explore more global features for better reconstruction. However, transformer modules bring in indispensable computational costs, and the coarse utilization of transformer in learned image compression cannot meet the coding efficiency. In this paper, we propose a novel graph-structured swin-transformer for learned image compression, shown in Figure 1 . We assume that the global receptive field of attention map should be sparse not dense, while the local neighboring correlations must be strong.

Recently, point-based point cloud geometry compression has attracted great attention due to its superior performance at low bit rates. However, lacking an efficient way to represent the local geometric correlation well, most existing methods [1, 2, 3] can hardly extract fine local features accurately. Thus it’s difficult for them to obtain high-quality reconstruction of local geometry of point clouds.

In this paper, we present a deep correlated image set compression scheme based on Distributed Source Coding(DSC) and multi-scale image fusion. As there exists strong correlation among images in a similar image set, we propose to utilize such correlation to generate side information at decoder side for each image in the set. Specifically, a reference structure of the image set is generated by building a minimum spanning tree according to the similarity between two images at encoder. With the reference structure, the side information of each image to be decoded can be generated based on the decoded reference image. And our network learns the correlation between an image and its side information in the training phase. Based on the principle of DSC, the side information can provide additional information such as rich details at decoder side. To make full use of the side information, the initially decoded image and the additional side information are fused at different scales. A decompressed image enhancement network is introduced to reduce the compression artifacts of the decoded images. Extensive experimental results compared with other mainstream methods validate the superior performance of our scheme in both terms of subjective and objective quality.

Motion estimation is a key component to remove the temporal redundancy in video compression. It is well-known that bi-directional motion estimation outperforms sequential motion estimation because of its capability to use both forward and backward reference frames. Previous approaches perform a coarse motion prediction operation to further remove motion spatial redundancy, which heavily rely on the regularity of the motion. However, most motions in natural video sequences are extremely complicated and objects usually move irregularly. To solve this problem, in this paper, we propose a fine motion prediction network by learning an importance map between the bi-directional references. Our designed network can generate more accurate prediction of the motion, yielding less residual. And it is also universal for videos with complex and irregular motions. Both objective and subjective quality results validate the effectiveness of our approach.

Removing undesirable reflections from a single image captured through a glass surface is of broad application to various image processing and computer vision tasks, but it is an ill-posed and challenging problem. Existing traditional single image reflection removal(SIRR) methods are often less efficient to remove reflection due to the limited description ability of handcrafted priors. State-of-the-art learning based methods often cause instability problems because they are designed as unexplainable black boxes. In this paper, we present an explainable approach for SIRR named model-guided unfolding network(MoG-SIRR), which is unfolded from our proposed reflection removal model with non-local autoregressive prior and dereflection prior. In order to complement the transmission layer and the reflection layer in a single image, we construct a deep learning framework with two streams by integrating reflection removal and non-local regularization into trainable modules. Extensive experiments on public benchmark datasets demonstrate that our method achieves superior performance for single image reflection removal.

The Spherical Measure Based Spherical Image Representation (SMSIR) has nearly uniformly distributed pixels in the spherical domain with effective index schemes. Based on SMSIR, the spherical wavelet transform can be efficiently designed, which can capture the spherical geometry feature in a compact manner and provides a powerful tool for spherical image compression. In this paper, we propose an efficient compression scheme for SMSIR images named Spherical Set Partitioning in Hierarchical Trees (S-SPIHT) using the spherical wavelet transform, which exploits the inherent similarities across the subbands in the spherical wavelet decomposition of a SMSIR image. The proposed S-SPIHT can progressively transform spherical wavelet coefficients into bit-stream, and generate an embedded compressed bit-stream that can be efficiently decoded at several spherical image quality levels. The most crucial part of our proposed S-SPIHT is the redesign of scanning the wavelet coefficients corresponding to different index schemes. We design three scanning methods, namely ordered root tree index scanning (ORTIS), dyadic index progressive scanning(DIPS) and dyadic index cross scanning(DICS)to efficiently reorganize the wavelet coefficients. These methods can effectively exploit the self-similarity between sub-bands and the fact that the high-frequency sub-bands mostly contain insignificant coefficients. Experimental results on widely-used datasets demonstrate that our proposed S-SPIHT outperforms the straightforward SPIHT for SMSIR images in terms of PSNR, S-PSNR and SSIM.

High-quality depth information has been increasingly used in many real-world multimedia applications in recent years. Due to the limitation of depth sensor and sensing technology, actually, the captured depth map usually has low resolution and black holes. In this paper, inspired by the geometric relationship between surface normal of a 3D scene and their distance from camera, we discover that surface normal map can provide more spatial geometric constraints for depth map reconstruction, as depth map is a special image with spatial information, which we called 2.5D image. To exploit this property, we propose a novel surface normal data guided depth recovery method, which uses surface normal data and observed depth value to estimate missing or interpolated depth values. Moreover, to preserve the inherent piecewise smooth characteristic of depth maps, graph Laplacian prior is applied to regularize the inverse problem of depth maps recovery and a graph Laplacian regularizer(GLR) is proposed. Finally, the spatial geometric constraint and graph Laplacian regularization are integrated into a unified optimization framework, which can be efficiently solved by conjugate gradient(CG). Extensive quantitative and qualitative evaluations compared with state-of-the-art schemes show the effectiveness and superiority of our method.

The rapid growth of surveillance videos poses a huge challenge for video coding technology. To make the best use of the special characteristics of surveillance videos, the prediction and coding methods with high-quality background picture (BG-picture) has been proposed. However, a large number of frames are always required for training in traditional background modeling methods, and too many bits are spent to encode the BG-picture. Therefore, we propose a double background based coding scheme for surveillance videos, in which two background frames are generated from the reconstructed frames and original frames respectively. Then residual frame between the original background and the reconstructed background is encoded to reduce the bit cost of the BG-picture. The experiments on surveillance videos shows that compared with HM14.0, the proposed method can achieve about 17 percent bit rate saving on average. Up to 40 percent bit rate saving can be observed on surveillance videos.

In this paper, we propose angle-normal images to reduce the number of normal component channels from three to two and present predicting the angle-normal images by the reconstructed geometry images. We implement the scheme on realistic meshes. Experimental results verify effectiveness of the proposed scheme. For geometry image codec, the proposed scheme outperforms up 1.78 dB PSNR gains, and average 0.57 dB PSNR gains. For normal image codec, the proposed scheme outperforms up 2.03 dB PSNR gains, and average 1.31 dB PSNR gains.

Summary form only given. Motion compensation is the key technique to reduce temporal redundancy in video coding. Interpolation filters are adopted to generate the inter frame prediction for motion compensation with fractional pixel accuracy. In existing video coding standards such as H.264/AVC and HEVC, a set of predefined interpolation filters is adopted in motion compensation. However, predefined interpolation filters cannot adapt to the video content, which may compromise the coding efficiency. In this paper, a content adaptive interpolation scheme is proposed for motion compensation. In the proposed scheme, a set of adaptive interpolation filters is derived for each frame as additional interpolation filters to minimize the inter prediction difference. Rate-distortion optimization is employed to choose between the predefined interpolation filters and the derived adaptive interpolation filters to achieve the best coding performance. The proposed scheme is implemented into the HM 12.1 software and. Experimental results show that the proposed scheme achieves 3.18 percent bit rate saving on average compared with HEVC.

Motion estimation plays an important role in video coding standards, such as H.264/AVC and HEVC. In this paper, we propose a multi-stage hash based motion estimation algorithm for HEVC, which enables hash based motion estimation for natural videos. In the proposed method, the prediction blocks significantly different from the current prediction unit will be eliminated in the motion estimation process. Locality sensitive hashing functions are used to measure the difference between the input block and predicted blocks. The proposed algorithm is implemented into the HM 12.0 software, and the simulation results show that the complexity of motion estimation is significantly reduced with negligible coding performance loss.

Screen contents with complex structure contain random combination of texts, graphics and camera-captured images, which makes them difficult to be compressed efficiently by traditional video codecs. In this paper, we propose a 2-D dictionary based scheme to exploit the repeated patterns on screen content. In the proposed scheme, the current block is predicted from the reconstructed region using a hash-based block searching scheme. A hierarchical two-level hash based searching scheme is designed to find the best matching block for each block. The first-level hash function is used to search the blocks similar to the current block in the constructed 2-D dictionary. The second-level hash function is used to update the 2-D dictionary, which filters out the identical blocks from the blocks found using the first-level hash function. The proposed scheme is incorporated into HEVC framework as an additional mode. Experimental results show that the proposed scheme achieves significantly coding performance improvements on screen contents compared with HEVC.