We present a method to super-resolve voxelized point clouds downsampled by a fractional factor, using lookup-tables (LUT) constructed from self-similarities from their own downsampled neighborhoods. The proposed method was developed to densify and to increase the precision of voxelized point clouds, and can be used, for example, as improve compression and rendering. We super-resolve the geometry, but we also interpolate texture by averaging colors from adjacent neighbors, for completeness. Our technique, as we understand, is the first specifically developed for intra-frame super-resolution of voxelized point clouds, for arbitrary resampling scale factors. We present extensive test results over different point clouds, showing the effectiveness of the proposed approach against baseline methods.

Point cloud compression has been studied in standard bodies and we are here concerned with the Moving Picture Experts Group video-based point cloud compression (V-PCC) solution. Plenoptic point clouds (PPC) is a novel volumetric data representation wherein points are associated with colors in all viewing directions to improve realism. It is sampled as a number ($N_{c}$) of attribute colors per point. We propose a new method for the efficient video-based compression of PPC that is backwards compatible with the existing single-color V-PCC decoder. V-PCC generates three image atlases which are encoded using an image/video encoder. We assume there may be a reference color which is to be encoded as the main payload. We generate$N_{c}+3$atlases and we produce$N_{c}$differential images against the reference color image. Those difference images are pixel-wise transformed using an$N_{c}$-point discrete cosine transform, generating$N_{c}$transformed atlases which are encoded, forming the secondary payload. Such secondary information is the plenoptic enhancement to the point cloud. If there is no reference attribute, we skip the differences and use the lowest frequency of the transformed atlases as the main payload. Results are presented that show an unrivaled performance of the proposed method.

Recently finalized, the MPEG Video-based Point Cloud Compression (V-PCC) standard leverages existing video codecs to compress point clouds. This approach relies on existing video coding hardware accelerators to enable its fast adoption. However, the processing steps to project 3D point clouds into 2D frames still have a considerable complexity. Thus, we propose two modifications to TMC2, V-PCC’s test model, considering the memory access pattern: one reducing unnecessary memory allocation and the other increasing data locality through a set of pre-processing steps. Our approach achieved up to 46.31% encoding self-time reduction without changing the resulting bitstream. This paper also provides an analysis of our implementation that may help future V-PCC codecs achieve real-time encoding.

We present a method to compress geometry information of point clouds that explores redundancies across consecutive frames of a sequence. It uses octrees and works by progressively increasing resolution of the octree. At each branch of the tree, we generate an approximation of the child nodes by a number of methods which are used as contexts to drive an arithmetic coder. The best approximation, i.e. the context that yields the least amount of encoding bits, is selected and the chosen method is indicated as side information for replication at the decoder. The core of our method is a context-based arithmetic coder in which a reference octree is used as reference to encode the current octree, thus providing 255 contexts for each output octet. The 255×255 frequency histogram is viewed as a discrete 3D surface and is conveyed to the decoder using another octree. We present two methods to generate the predictions (contexts) which use adjacent frames in the sequence (inter-frame) and one method that works purely intra-frame. The encoder continuously switches the best mode among the three and conveys such information to the decoder. Since an intra-frame prediction is present, our coder can also work in purely intra-frame mode, as well. Extensive results are presented to show the method's potential against many compression alternatives for the geometry information in dynamic voxelized point clouds.

We propose a mixed-resolution point-cloud representation and an example-based super-resolution framework, from which several processing tools can be derived, such as compression, denoising and error concealment. By inferring the high-frequency content of low-resolution frames based on the similarities between adjacent full-resolution frames, the proposed framework achieves an average 1.18 dB gain over lowpass versions of the point-cloud, for a projection-based distortion metric [1], [2].

3D and free-viewpoint video has been slowly adopting a solid representation, such as using meshes and point-clouds. Among other characteristics, meshes provide direct surface representation, while point-cloud processing requires less computation. Points in the cloud are minimally represented by their geometry (3D position) and color. A common point-cloud geometry compression method is the octree representation, which acts on individual frames and can be further compressed by entropy encoding. This paper presents a lossless intra-frame compression method for point-cloud geometry, which uses the octree structure to provide better contexts for entropy coding. Results show that the proposed solution offers state-of-the-art performance, with an average rate reduction of 29% compared to the octree representation.

We present a context-driven method to encode nodes of an octree, which is typically used to encode the point-cloud geometry. Instead of using one bit per node of the tree, the context allows for deriving probabilities for that node based on distances of the actual voxel to the voxels in a reference point cloud. Accurate probabilities of the node state allow for the use of an arithmetic coder to reduce the bit rate. Results point to potentially large reductions in rate if there is a good model from which to derive the context, i.e., one can get large reductions if the reference-cloud geometry is close enough to the one being encoded.

3D and free-viewpoint video has been moving towards solid-scene representation using meshes and point clouds. Point-cloud processing requires much less computation and the points in the cloud are minimally represented by their geometry (3D position) and color. A common point-cloud geometry compression method is the octree representation, which acts on individual frames. We present a lossless inter-frame compression method for point-cloud geometry, by reordering each octree based on previous frames prior to entropy coding. Results show that compared to the octree representation, the proposed solution offers an average rate reduction of 30%, while entropy-coding of the octree yields an average rate reduction of 24%.

Face-recognition biometric systems have been shown unreliable under the presence of face-spoofing images, creating the need for automatic spoofing detection. In this paper, the effect of image compression degradation on face-spoofing detection is evaluated, based on an algorithm that searches for Moiré patterns due to the overlap of the digital grids, through peak detection in the frequency domain. The proposed spoofing-detection algorithm performance is evaluated subject to H.264/AVC and JPEG image compression for an image database of facial shots under several conditions.

Biometric systems based on face recognition have been shown unreliable under the presence of face-spoofing images. Hence, automatic solutions for spoofing detection became necessary. In this paper, face-spoofing detection is proposed by searching for Moiré patterns due to the overlap of the digital grids. The conditions under which these patterns arise are first described, and their detection is proposed which is based on peak detection in the frequency domain. Experimental results for the algorithm are presented for an image database of facial shots under several conditions.

We present an example-based approach to general enhancement of degraded video frames. The method relies on building a dictionary with non-degraded parts of the video and to use such a dictionary to enhance the degraded parts. The image degradation has to originate from a “repeatable” process, so that the dictionary image patches (blocks) are equally degraded, thus originating a dictionary with degraded blocks and their residues (differences in between degraded and original blocks). Once a match is found between a degraded block in the video and a degraded block in the dictionary, the associated residue of the latter is soft-added to the block of the former. The method is a generalization of the method for example-based super-resolution. Results are presented to demonstrate the applicability of the method to many scenarios.

We propose a mixed-resolution coding architecture for stereo color-plus-depth images, where encoding is performed at a low resolution, except for one of the color images. Super-resolution methods are proposed for the depth maps and for the low-resolution color image at the decoder side. Experiments are carried out for several real and synthetic images and reveal a reduction in complexity at the encoder associated with average PSNR gains of the super-resolved low-resolution color image with respect to regular interpolation.

In stereoscopic and multiview video, binocular suppression theory states that the visual subjective quality of 3-D experience is not much affected by asymmetrical blurring of the individual views. Based on these studies, mixed-resolution frameworks applied for multiview systems offer great data-size reduction without incurring in significant quality degradation in 3-D video applications. However, it is interesting to recover high-frequency content of the blurred views, to reduce visual strain due to long-term exposure and to make the system suitable for free-viewpoint television. In this paper, we present a novel super-resolution technique, in which low-resolution views are enhanced with the aid of high-frequency content from neighboring full-resolution views, and the corresponding depth information for all views. Occlusions are handled by checking the consistency between views. Tests for synthetic and real image data in stereo and multiview cases are presented, and results show that significant objective quality gains can be achieved without any extra side information.

Encoding video sequences is a computation-demanding task in high-performance codecs. Optimizing this stage may result in a substantial encoding speed-up. In this paper, we propose a faster approach to encode sequences with the H.264/AVC codec, using mixed-resolution. By having some of the frames down-sampled, the overall computation is reduced, without greatly affecting the rate-distortion performance, which remains evaluated using the sequence's original resolution. In addition to this approach, a method which regards only the most frequent prediction modes of a frame is incorporated, so that the combined complexity reduction can be significant with negligible performance losses.

This paper presents a transform-based approach to semi-super resolution. The idea of semi-super resolution is to enhance low-resolution frames from a video sequence encoded with different resolutions among frames. The proposed framework uses a DCT-based down-sampling method at the encoder process. At the decoder, DCT-based up-sampling plus high-frequency information from adjacent frames in full resolution are used to enhance interpolated low-resolution frames. The results show an increase of the overall quality without using any extra information from the encoder. The proposed technique can also outperfom previous semi-super resolution methods operating in pixel domain.

The joint usage of low- and full-resolution images in multiview systems provides an attractive opportunity for data size reduction while maintaining good quality in 3D applications. In this paper we present a novel application of a super-resolution method for usage within a mixed resolution multiview setup. The technique borrows high-frequency content from neighboring full resolution images to enhance particular low-resolution views. Occlusions are handled through matching of low-resolution images. Both the stereo and the more general multiview cases are considered using the multiview video-plus-depth format. Results demonstrate significant gains in PSNR and in visual quality for test sequences.

A new intra-prediction mode for the H.264/AVC standard is proposed. Each pixel within a block is predicted by a weighted sum of its neighbours, according to anNth order Markov linear model. The weights are obtained through a least-squares estimate from reconstructed data in the neighbouring blocks, so that no overhead is necessary to convey the weights to the decoder. Results show significant improvements in H.264/AVC compression for images that are rich in directional structures, and moderate improvements for the other images and sequences tested.