Holographic displays are a promising technology for delivering immersive, true 3D visualization in virtual and augmented reality applications. However, generating high-fidelity phase-only holograms remains challenging, especially with the demand for efficient compression to handle the substantial data inherent in high-resolution holographic streaming. Existing techniques often struggle to balance the trade-off between optical display quality and compression efficiency, and jointly optimizing these aspects is still in its infancy. This work presents a learning-empowered multi-color hologram compression scheme that utilizes a pre-trained, camera-calibrated wave propagation model, especially for unfiltered holographic display configurations with compact form factors. In particular, the inter-color processing leverages the inherent redundancy across color channels, allowing for efficient compression. By incorporating the learned camera-calibrated wave propagation model into our training process, we can achieve superior optical display quality and compression rates. Experiments demonstrate that our method realizes a reduction in bits per pixel (bpp) of 44% to 74% over representative baselines at the same quality level. We envision the proposed compressive hologram synthesis scheme establishing a new benchmark for high-fidelity holographic reconstruction at lower bitrates, marking a significant advance towards the deployment of holography-empowered visual media systems.

Holographic displays offer the promise of realistic 3D visualization for virtual and augmented wearable solutions. Nevertheless, existing computer-generated holography (CGH) methods often struggle with either a high computational burden or limited display realism. While the emerging cloud-edge computing mechanism can enable the real-time streaming of holograms, classic image compression techniques struggle to efficiently encode and decode the substantial high-frequency information inherent in hologram data. In light of these challenges, we present a display-aware and lightweight CGH framework, leveraging implicit neural representations (INRs) and camera-calibrated wave propagation, to generate and compress high-fidelity phase-only holograms. Specifically, our approach interprets hologram generation as a continuous function approximation problem, enabling the network, with reduced parameters, to effectively learn the inherent periodicity and high-frequency components of 2D and 3D hologram data. To enable efficient deployment, we further incorporate quantization-aware training, followed by entropy coding. Experimental results evaluated on an unfiltered holographic display prototype demonstrate that the proposed INR-CGH retains image quality comparable to that of existing optimization-based methods in both 2D and 3D scenarios. In addition, our compact INR representation achieves up to 11× compression rate with minimal quality degradation and can be further reduced via quantization-aware training. The resulting model enables ≥250 fps in decoding speed, paving the way towards edge holography.

Recently, neural network-based approaches for hologram generation and compression have gained popularity as they allow for efficient inference on GPUs without the need for iterative optimization required in traditional methods. In this paper, we introduce Neural Holographic Video Compression (NHVC), an end-to-end trainable and scalable model designed for high-quality phase hologram video generation and compression. NHVC consists of an auto-encoder-based phase hologram generator, a latent coder and-two hyper-prior coders. For each input image, the latent features are extracted through the encoder part of the phase generator and then entropy coded at the shared latent coder based on the hyper-prior information. The two hyper-prior coders employ a spatial and a spatio-temporal entropy model for I-frames and P-frames, respectively. With this architecture, our NHVC can offer task-scalability, allowing a single trained model to serve as a phase hologram generator, phase hologram image compressor, or phase hologram video compressor as required.Experimental results on phase hologram video compression with UVG dataset show that our model outperforms ‘HoloNet + VVC’ by 75.6% BD-Rate reduction, with modest 2K encoding and decoding speeds (5 fps and 12 fps, respectively). For the phase hologram video generation task, our model showed much higher-quality (almost 42dB PSNR) reconstruction using the UVG dataset, while the previous neural generation model HoloNet provides at most 36dB reconstruction quality. We also provide an extensive experimental study on several important design questions such as the need for quadruple extension (QE) in the neural compression model, the feasibility of motion estimation in the phase domain, and an alternative, the need for increasing receptive field to learn better phase features, and variable rate support with a single trained model. It is noteworthy that our model is the first and best neural phase video compression model providing such high-quality reconstruction and task-scalability.