Thanks to plenoptic cameras, rich information on the radiance of a scene can be conveniently captured without heavy devices like camera arrays. However, rendering techniques are needed to generate views for human visual perception. Existing patch extraction-based rendering techniques can generate views from the lenslet images captured by plenoptic cameras, but they suffer from the inherent problem of artifacts and the limited views to be rendered. In this paper, we present a new view rendering technique from plenoptic 2.0 camera-captured lenslet image by using 3-dimensional gaussian splatting (3DGS). At its first step, the reference lenslet converter (RLC) provided by MPEG LVC AhG, one of the existing patch extraction methods, generates initial views with the help of estimated disparity between adjacent micro images in the lenslet image. At its second step, the 3DGS generates the final views after being trained by the initial views. The rendering results obtained by the proposed 2-step approach show significantly fewer artifacts in the rendered views than the patch stitching process of the existing method.

This paper introduces a novel intra-prediction scheme for coding plenoptic video which can effectively exploit large correlation between current and neighboring macropixel images. While the intra block copy method is well recognized as a promising coding tool for plenoptic video, it has fundamental issues like much searching time for block vectors (BVs) and more bits to encode these BVs into a bitstream. Our method can effectively solve them by pre-defining the prediction candidates to save encoding time and signaling only the index of prediction location instead of BVs to reduce the overhead bits for encoding BVs. Compared to HEVC, our method is experimentally shown to achieve an average bitrate gain of about $19.70 \%$ and $11.99 \%$ respectively under the AI-Main and RA-Main conditions. Moreover, better trade-off can be made between complexity and coding performance than existing methods.

In this paper we introduce a novel approach to better utilize the intra block copy (IBC) prediction tool in encoding lenslet light field video (LFV) captured using plenoptic 2.0 cameras. Although the IBC tool has been recognized as promising for encoding LFV content, its fundamental limit due to its original design rooted for encoding conventional videos suggests slight modification possibility to better suit the property of LFV content. Observing the inherently large amount of repetitive image patterns due to the microlens array (MLA) structure of plenoptic cameras, several techniques are suggested in this paper to enhance the IBC coding tool itself for more efficiently encoding LFV contents. Our experimental results demonstrate that the proposed method significantly enhances the IBC coding performance in case of encoding LFV contents while concurrently reducing encoding time.

Light field (LF) rescaling is indispensable in accommodating different LF image resolutions for different applications. Unlikely most recent studies which only execute learned LF upscaling from a predefined downscaling method, we propose a novel LF rescaling framework by jointly optimizing learned LF downscaling and upscaling as a combined task. Specifically, our light field rescaling network (LFRN) simultaneously extracts features from different 2D subspaces of LF data (e.g., spatial-angular and epipolar subspaces) to fully handle 4D LF image information. Our newly designed attention fusion module (AFM) adaptively combines these two data features based on learnable embedding weights. Due to joint optimization of the learned LF downscaling and upscaling tasks, our LFRN method can achieve significant performance gain in both objective and subjective visual qualities compared to conventional predefined downscaling with learned LF upscaling task.

This paper proposes a novel hybrid light field (LF) denoising method which is based on a convolutional neural network (CNN) designed to reflect the characteristic of LF image in both pixel and frequency domains. Noting that the image noise usually has much high-frequency energy, the proposed network is designed to operate in a transform domain in two stages. At the first stage, energy compaction of spatial-angular information of LF image is sought by 4D-DCT separated transform which can achieve better energy compaction than 2D-DCT applied separately in the spatial and angular domain. The transformed LF is decomposed into different frequency components and each frequency component is recovered progressively. Subsequently, we reshape and convert different frequency components into pixel domain to perform the next refinement step for which a residual spatial-angular block (RSAB) is proposed to handle the 4D LF structure in the pixel domain. Extensive experimental results on different noisy datasets confirm the effectiveness of our proposed method compared to state-of-the-art methods in both objective and subjective quality.

Multi-focused plenoptic images possess many special characteristics related to the micro-images (MIs) array, which are expected to be useful in further increasing its compression performance. Those special characteristics come from the much overlap and sharpness variance among its micro-images, and proper handling of such properties can lead to better patch-based prediction. In this paper, for multi-focused plenoptic image data, we design a new prediction model taking into account the disparity shift constraint coming from the overlaps and the sharpness variation. Experiment results show coding gain respectively of 21% over the HEVC Intra and 27% when the proposed method is combined with the Intra Block Copy (IBC) tool which is reported very effective in plenoptic image coding.

Plenoptic images and videos bearing rich information demand a tremendous amount of data storage and high transmission cost. While there has been much study on plenoptic image coding, investigations into plenoptic video coding have been very limited. We investigate the motion compensation (or so-called temporal prediction) for plenoptic video coding from a slightly different perspective by looking at the problem in the ray-space domain instead of in the conventional pixel domain. Here, we develop a novel motion compensation scheme for lenslet video under two sub-cases of ray-space motion, that is, integer ray-space motion and fractional ray-space motion. The proposed new scheme of light field motion-compensated prediction is designed such that it can be easily integrated into well-known video coding techniques such as HEVC. Experimental results compared to relevant existing methods have shown remarkable compression efficiency with an average gain of 20.03% and 21.76% respectively under "Low delayed B " and "Random Access" configurations of HEVC.

In raw plenoptic video, the optimal motion searching points mostly follow the hexagonal structure of micro-images. Based on this understanding, we propose a new motion vector resolution, namely the hexagonal lattice (HL) resolution which reflects micro-image structure. The HL resolution can be efficiently represented by HL basis. A study in this paper shows that motion vectors are highly concentrated at hexagonal lattice points, leading to use of the proposed resolution in the context of video compression. In this regard, we demonstrate the compression benefit brought by estimating motion vectors at HL resolution in the VVC codec.

This paper proposes a new downsampling-based light field video coding (D-LFVC) framework whose success relies on how to design an effective restoration method that can remove artifacts brought by both downsampling and compression. Since light field (LF) video is of high dimensionality data, the restoration methods designed for conventional 2D video are sub-optimal solutions for our D-LFVC. In this regard, we design a new restoration network, named "LF-QEN," for our D-LFVC framework. Specifically, the network contains three different feature extractor modules, allowing us to simultaneously exploit information from different kinds of 4D LF representation: spatial, angular, and epipolar image information. Our experimental results show that, compared to compression by HEVC-SCC standard, the proposed framework can obtain not only nearly 50% bitrate savings but also can significantly enhance the quality of decoded LF video.

In deep convolutional neural networks (DCNNs) for single image super-resolution (SISR), the dense and residual feature refinement helps to stabilize the training network and enriches the feature values. However, most SISR networks do not fully exploit the rich feature information in the hierarchical dense residual connections, thus achieving relatively low performance. Besides, in many cases, a large model is not feasible to deploy on mobile or embedded devices. By exploiting the hierarchical dense residual learning, this paper proposes a fast and efficient hierarchical dense residual network (HDRN) to solve these problems. Specifically, we develop a dense compact residual group (DCRG), consisting of several compact residual blocks (CRB), which helps to increase the reusable feature capability. Our experimental results confirm that the proposed HDRN achieves better trade-off between the performance and computational costs than those state-of-the-art lightweight SISR methods.