This paper introduces a novel intra-prediction scheme for coding plenoptic video which can effectively exploit large correlation between current and neighboring macropixel images. While the intra block copy method is well recognized as a promising coding tool for plenoptic video, it has fundamental issues like much searching time for block vectors (BVs) and more bits to encode these BVs into a bitstream. Our method can effectively solve them by pre-defining the prediction candidates to save encoding time and signaling only the index of prediction location instead of BVs to reduce the overhead bits for encoding BVs. Compared to HEVC, our method is experimentally shown to achieve an average bitrate gain of about $19.70 \%$ and $11.99 \%$ respectively under the AI-Main and RA-Main conditions. Moreover, better trade-off can be made between complexity and coding performance than existing methods.

In this paper we introduce a novel approach to better utilize the intra block copy (IBC) prediction tool in encoding lenslet light field video (LFV) captured using plenoptic 2.0 cameras. Although the IBC tool has been recognized as promising for encoding LFV content, its fundamental limit due to its original design rooted for encoding conventional videos suggests slight modification possibility to better suit the property of LFV content. Observing the inherently large amount of repetitive image patterns due to the microlens array (MLA) structure of plenoptic cameras, several techniques are suggested in this paper to enhance the IBC coding tool itself for more efficiently encoding LFV contents. Our experimental results demonstrate that the proposed method significantly enhances the IBC coding performance in case of encoding LFV contents while concurrently reducing encoding time.

Light field (LF) rescaling is indispensable in accommodating different LF image resolutions for different applications. Unlikely most recent studies which only execute learned LF upscaling from a predefined downscaling method, we propose a novel LF rescaling framework by jointly optimizing learned LF downscaling and upscaling as a combined task. Specifically, our light field rescaling network (LFRN) simultaneously extracts features from different 2D subspaces of LF data (e.g., spatial-angular and epipolar subspaces) to fully handle 4D LF image information. Our newly designed attention fusion module (AFM) adaptively combines these two data features based on learnable embedding weights. Due to joint optimization of the learned LF downscaling and upscaling tasks, our LFRN method can achieve significant performance gain in both objective and subjective visual qualities compared to conventional predefined downscaling with learned LF upscaling task.

This paper proposes a novel hybrid light field (LF) denoising method which is based on a convolutional neural network (CNN) designed to reflect the characteristic of LF image in both pixel and frequency domains. Noting that the image noise usually has much high-frequency energy, the proposed network is designed to operate in a transform domain in two stages. At the first stage, energy compaction of spatial-angular information of LF image is sought by 4D-DCT separated transform which can achieve better energy compaction than 2D-DCT applied separately in the spatial and angular domain. The transformed LF is decomposed into different frequency components and each frequency component is recovered progressively. Subsequently, we reshape and convert different frequency components into pixel domain to perform the next refinement step for which a residual spatial-angular block (RSAB) is proposed to handle the 4D LF structure in the pixel domain. Extensive experimental results on different noisy datasets confirm the effectiveness of our proposed method compared to state-of-the-art methods in both objective and subjective quality.

Multi-focused plenoptic images possess many special characteristics related to the micro-images (MIs) array, which are expected to be useful in further increasing its compression performance. Those special characteristics come from the much overlap and sharpness variance among its micro-images, and proper handling of such properties can lead to better patch-based prediction. In this paper, for multi-focused plenoptic image data, we design a new prediction model taking into account the disparity shift constraint coming from the overlaps and the sharpness variation. Experiment results show coding gain respectively of 21% over the HEVC Intra and 27% when the proposed method is combined with the Intra Block Copy (IBC) tool which is reported very effective in plenoptic image coding.

Plenoptic images and videos bearing rich information demand a tremendous amount of data storage and high transmission cost. While there has been much study on plenoptic image coding, investigations into plenoptic video coding have been very limited. We investigate the motion compensation (or so-called temporal prediction) for plenoptic video coding from a slightly different perspective by looking at the problem in the ray-space domain instead of in the conventional pixel domain. Here, we develop a novel motion compensation scheme for lenslet video under two sub-cases of ray-space motion, that is, integer ray-space motion and fractional ray-space motion. The proposed new scheme of light field motion-compensated prediction is designed such that it can be easily integrated into well-known video coding techniques such as HEVC. Experimental results compared to relevant existing methods have shown remarkable compression efficiency with an average gain of 20.03% and 21.76% respectively under "Low delayed B " and "Random Access" configurations of HEVC.

In raw plenoptic video, the optimal motion searching points mostly follow the hexagonal structure of micro-images. Based on this understanding, we propose a new motion vector resolution, namely the hexagonal lattice (HL) resolution which reflects micro-image structure. The HL resolution can be efficiently represented by HL basis. A study in this paper shows that motion vectors are highly concentrated at hexagonal lattice points, leading to use of the proposed resolution in the context of video compression. In this regard, we demonstrate the compression benefit brought by estimating motion vectors at HL resolution in the VVC codec.

This paper proposes a new downsampling-based light field video coding (D-LFVC) framework whose success relies on how to design an effective restoration method that can remove artifacts brought by both downsampling and compression. Since light field (LF) video is of high dimensionality data, the restoration methods designed for conventional 2D video are sub-optimal solutions for our D-LFVC. In this regard, we design a new restoration network, named "LF-QEN," for our D-LFVC framework. Specifically, the network contains three different feature extractor modules, allowing us to simultaneously exploit information from different kinds of 4D LF representation: spatial, angular, and epipolar image information. Our experimental results show that, compared to compression by HEVC-SCC standard, the proposed framework can obtain not only nearly 50% bitrate savings but also can significantly enhance the quality of decoded LF video.

In deep convolutional neural networks (DCNNs) for single image super-resolution (SISR), the dense and residual feature refinement helps to stabilize the training network and enriches the feature values. However, most SISR networks do not fully exploit the rich feature information in the hierarchical dense residual connections, thus achieving relatively low performance. Besides, in many cases, a large model is not feasible to deploy on mobile or embedded devices. By exploiting the hierarchical dense residual learning, this paper proposes a fast and efficient hierarchical dense residual network (HDRN) to solve these problems. Specifically, we develop a dense compact residual group (DCRG), consisting of several compact residual blocks (CRB), which helps to increase the reusable feature capability. Our experimental results confirm that the proposed HDRN achieves better trade-off between the performance and computational costs than those state-of-the-art lightweight SISR methods.

The motion estimation which plays an important role in video coding requires much computation for encoding. In this paper, from the ray motion characteristics in the lenslet plenoptic video, we derive a new motion search model and propose a fast and efficient microlens-based motion search method. Theoretical analysis and experimental results have verified the new model and demonstrated its efficiency in search. Under the HEVC random-access configuration, we achieve not only substantial encoding time reduction (56.7%), but also bitrate saving of 1.3% on average compared to relevant existing works. Under the low delay configuration, the performances are 23.3% and 2.3%, respectively for encoding time reduction and bitrate saving.

This paper proposes a novel data cost that combines spatial and spectral entropies to handle the occlusion problem in the light field depth estimation. In previous works, the spatial entropy data cost has been demonstrated to reduce the effect of occluded pixels in an angular patch (i.e., micro-lens pixel) and to yield an accurate depth value in the presence of occlusion. However, our observation notes that the spatial entropy data cost metric is less reliable when the angular resolution becomes smaller as in light field images. In this paper, we propose a new data cost which integrates a proposed spectral entropy data cost with the spatial entropy data cost. An initial depth map which is estimated using the proposed new data cost is further optimized by the standard graph-cut algorithm and filtered by using an edge-preserving filter. Experimental results have confirmed the effectiveness of the proposed method which achieves more accurate depth values even when the angular resolution becomes smaller.

The increasing prevalence of VR/AR as well as the expected availability of Light Field (LF) display soon call for more practical methods to transmit LF image/video for services. In that aspect, the LF video coding should not only consider the compression efficiency but also the view random-access capability (especially in the multi-view-based system). The multi-view coding system heavily exploits view dependencies coming from both inter-view and temporal correlation. While such a system greatly improves the compression efficiency, its view random-access capability can be much reduced due to so called "chain of dependencies." In this paper, we first model the chain of dependencies by a tree, then a cost function is used to assign an importance value to each tree node. By travelling from top to bottom, a node of lesser importance is cut-off, forming a pruned tree to achieve reduction of random-access complexity. Our tree pruning method has shown to reduce about 40% of random-access complexity at the cost of minor compression loss compared to the state-of-the-art methods. Furthermore, it is expected that our method is very lightweight in its realization and also effective on a practical LF video coding system.