It’s difficult to effectively remove redundancy in Li-DAR point clouds due to their extremely sparse and nonuniform distribution. Taking advantage of both octree-based methods and voxel-based schemes, we propose to design an effective temporal-spatial context to compress the sequence octree-structured point cloud data into a more compact bitstream. In this paper, we first build a temporal-spatial multiscale context for the deep learning entropy model. It further utilize the correlation of sequential point cloud data from both the spatial domain and temporal domain. In terms of spatial context, we design a hierarchical dependency in an octree to encode the occupancy information of each non-leaf octree node into a bitstream. We propose to further group the nodes according to their octant which effectively expands the context receptive field. In terms of temporal context, the KNN algorithm is applied to explore the most relative context with the strongest dependency in the temporal domain. Finally, we design a voxel re-localization network to convert the discrete voxels into refined 3D points, which makes up for the coordinate loss in the process of generating an octree. The quantitative evaluation shows that our method outperforms state-of-the-art baselines with saving most bitrate on KITTI Odometry dataset, and achieving the best reconstreuction benefit by the designed refinement module.

Point clouds captured in real-world applications are often in-complete due to the limited sensor resolution, viewpoint, and occlusion. Therefore, recovering the completion point clouds from incomplete ones becomes an important work in many practical applications. However, most of previous work only focus on the point-to-point relationship between the reconstructed point cloud and groundtruth(GT), not fully exploring their local geometry relationship, which results in the inaccu-rate and nonuniform local details. To solve this problem, we propose a novel local geometry preserving point cloud com-pletion network(LGP-Net). By promoting the consistency of local geometry features between the reconstruction and GT, the proposed LGP-Net can preserve more accurate local de-tails. To further explore the local geometry correlation at dif-ferent scales, a multi-scale local geometry consistency is also proposed. Moreover, the consistency between the features at different scales are proposed to exploit the correlation of features under different resolutions. Quantitative and qualitative results on the benchmark dataset demonstrate that our LGP-Net achieves superior performance over several state-of-the-art methods significantly.

The edge information plays a key role in the restoration of a depth map. Most conventional methods assume that the color image and depth map are consistent in edge areas. However, complex texture regions in the color image do not match exactly with edges in the depth map. In this paper, firstly, we point out that in most cases the consistency between normal map and depth map is much higher than that between RGB-D pairs. Then we propose a dual normal-depth regularization term to guide the restoration of depth map, which constrains the edge consistency between normal map and depth map back and forth. Moreover, considering the bimodal characteristic of weight distribution that exists in depth discontinuous areas, a reweighted graph Laplacian regularizer is proposed to promote this bimodal characteristic. And this regularization is incorporated into a unified optimization framework to effectively protect the piece-wise smoothness(PWS) characteristics of depth map. By treating depth image as graph signal, the weight between two nodes is adapted according to its content. The proposed method is tested for both noise-free and noisy cases, and is compared against the state-of-the-art methods on both synthesis and real captured datasets. Extensive experimental results demonstrate the superior performance of our method compared with most state-of-the-art works in terms of both objective and subjective quality evaluations. Specifically, our method is more effective on edge areas and more robust to noises.

Depth maps acquired by 3D cameras usually suffer from low resolution and insufficient quality, which makes it difficult to be directly used in visual depth perception and 3D reconstruction. To handle this problem, we propose a novel multi-direction dictionary and joint regularization model for high quality depth recovery. To enhance the sparsity, image is divided into classified patches according to the same geometrical direction and a compact dictionary is trained within each class. Then for a patch to be coded, the most relevant dictionary can be selected according to its geometrical direction. We further introduce two regularization terms into the reconstruction model. One is the anisotropic total variation (TV) defined by the local gradients of color/depth pair. The other is the nonlocal similarity to provide nonlocal constraint to the local structure. Experimental results demonstrate that our method outperforms other state-of-the-art methods in terms of both subjective quality and objective quality.

The edge information plays key role in the restoration of depth map. Most conventional methods assume that the RGB-D pairs are consistent in edge areas. In this paper, firstly, we point out that in most cases the consistency between normal map and depth map(N-D pairs) are much higher than that be-tween RGB-D pairs. Then we propose a dual regularization term to guide the restoration of depth map, which constrains the consistency between N-D pairs back and forth. Moreover, a reweighted graph Laplacian prior is incorporated into a unified optimization framework to effectively protect piece-wise smoothness(PWS) characteristics of depth map. By treating depth maps as graph signals, the weight between two nodes is adapted according to its content. Extensive experimental results demonstrate the superior performance of our method compared with other state-of-the-art works in terms of objective and subjective quality evaluations.

High-quality depth information has been increasingly used in many real-world multimedia applications in recent years. Due to the limitation of depth sensor and sensing technology, actually, the captured depth map usually has low resolution and black holes. In this paper, inspired by the geometric relationship between surface normal of a 3D scene and their distance from camera, we discover that surface normal map can provide more spatial geometric constraints for depth map reconstruction, as depth map is a special image with spatial information, which we called 2.5D image. To exploit this property, we propose a novel surface normal data guided depth recovery method, which uses surface normal data and observed depth value to estimate missing or interpolated depth values. Moreover, to preserve the inherent piecewise smooth characteristic of depth maps, graph Laplacian prior is applied to regularize the inverse problem of depth maps recovery and a graph Laplacian regularizer(GLR) is proposed. Finally, the spatial geometric constraint and graph Laplacian regularization are integrated into a unified optimization framework, which can be efficiently solved by conjugate gradient(CG). Extensive quantitative and qualitative evaluations compared with state-of-the-art schemes show the effectiveness and superiority of our method.