Depth from focus is a simple and effective methodology for retrieving the scene geometry from a hologram when used with the appropriate focus measure and patch size. However, fixing those parameters for every sample may not be the right choice, as different scenes can be composed with various types of textures. In this work, we propose a self-supervised learning methodology for fusing the depth maps produced using different focus measures with variable patch sizes applied to the holographic reconstruction volume. Experimental results show that fusing depth information produces more accurate and smoother depth maps, which can be directly used for alternative tasks such as motion estimation.

Despite the growing interest for Holography, there is a lack of publicly available three-dimensional hologram sequences for the evaluation of video codecs with inter-frame compression mechanisms such as motion estimation and compensation. In this paper, we report the first large-scale dataset containing 18 holographic videos computed with three different resolutions and pixel pitches. By providing the color and depth images corresponding to each hologram frame, our dataset can be used in additional applications such as the validation of 3D scene geometry retrieval or deep learning-based hologram synthesis methods. Altogether, our dataset comprises 5400 pairs of RGB-D images and holograms, totaling more than 550 GB of data.

We present a wearable full-color holographic augmented reality headset with binocular vision support and real-time embedded hologram calculation. Contrarily to most previously proposed prototypes, our headset employs high-speed amplitude-only microdisplays and embeds a compact and lightweight electronic board to drive and synchronize the microdisplays and light source engines. In addition, to enable a standalone usage of the headset, we developed a real-time hologram rendering engine capable of computing full-color binocular holograms at over 35 frames per second on a NVIDIA Jetson AGX Orin embedded platform. Finally, we provide a comparison of the efficiency of laser diodes and superluminescent diodes for the reduction of speckle noise, which greatly affects the reconstructed image’s quality. Experimental results show that our prototype enables full-color holographic images to be reconstructed with accurate focus cues and reduced speckle noise in real-time.

Unlike conventional images and videos, digital holograms contain large amounts of data with very low redundancy. Consequently, current communication networks may be not able to meet the bandwidth requirements for hologram transmission in reasonable time. To enable practical streaming of holographic contents, we propose a progressive coding method that combines quality scalability with viewpoint scalability. From a Gabor wavelets decomposition of the hologram, the server starts by selecting the coefficients corresponding to the user's viewpoint. Then, the selected coefficients are encoded progressively according to their importance for the reconstructed view. Experimental results reveal that our approach outperforms conventional scalable codecs and enables the streaming of holographic data with a better quality of experience.

We propose a fast Computer-Generated Hologram (CGH) computation method based on a hybrid point-source/wave-field approach. Whereas previously proposed methods tried to reduce the computational complexity of the point-source or the wave-field approaches independently, our method uses the two approaches together and therefore takes advantages from both of them. The algorithm consists of three steps. First, the 3D scene is sliced into several depth layers parallel to the hologram plane. Then, for each layer, we compute the complex wave scattered by this layer either using a wave-field or a point-source approach according to a threshold criterion on the number of points within the layer. Finally, we sum up the complex waves scattered by all the depth layers in order to obtain the final CGH. Experimental results reveal that this combination of approaches does not produce any visible artifact and outperforms both the point-source and wave-field approaches.

In this paper, we present a framework for networked hologram adaptive transmission. Depending on the observer position, a subset of hologram data is transmitted and displayed. Wavelet decomposition and pruning of wavelet coefficients according to the user position allows local diffractive pattern extraction. The proposed framework has been validated on an experimental set-up involving a kinect sensor for viewer position estimation.

Integral imaging is an attractive auto-stereoscopic three dimensional (3D) technique for next generation 3DTV. To improve its video quality, new techniques are required to effectively compress the huge volume of integral image (II) data. In this paper, a new compression method implemented by multi-view video coding (MVC) is provided and used for sub-images (SI). SI is an alternative form of 2D image transformed from original II. Each SI represents the 3D scene from parallel viewing directions and contains superior compression capabilities than original captured elemental images (EI). For this reason, we consider arranging the group of SIs as the format of multi-view video (MVV) and then encode the generated MVV by MVC standard. Experimental results show that our proposed compression approach improves the compression efficiency when compared to the traditional MPEG-4/AVC compression method for II.

Although compression of 3D content has been a widely studied topic, adaptive streaming of large virtual scenes is progressively becoming the standard way to share collaborative environments, due to the growing complexity of the shared data. The classical 'download and play' paradigm associated to standalone compression is leaving place to interactive refinements of regions-of-interest, provided given polygon budget and network capability. With the increasing number of applications and potential clients sharing reusable data, the traditional client / server architecture is likely to be over passed by less server-centered protocols, in order to distribute the complex tasks of identifying the selected data to be streamed. In this paper, we present the design and performances of a platform exploiting peer-to-peer transmission and show how the whole setting can be implemented using the MPEG-4 AFX standards.

In this paper, we present a three-dimensional (3D) model-based video coding scheme for streaming static scene video in a compact way but also enabling time and spatial scalability according to network or terminal capability and providing 3D functionalities. The proposed format is based on encoding the sequence of reconstructed models using second-generation wavelets, and efficiently multiplexing the resulting geometric, topological, texture, and camera motion binary representations. The wavelets decomposition can be adaptive in order to fit to images and scene contents. To ensure time scalability, this representation is based on a common connectivity for all 3D models, which also allows straightforward morphing between successive models ensuring visual continuity at no additional cost. The method proves to be better than previous methods for video encoding of static scenes, even better than state-of-the-art video coders such as H264 (also known as MPEG AVC). Another application of our approach are smoothing camera path for suppression of jitter from hand-held acquisition and the fast transmission and real-time visualization of virtual environments obtained by video capture, for virtual or augmented reality and interactive walk-through in photo-realistic 3D environments around the original camera path

This paper presents an efficient and scalable coding scheme for transmitting a stream of 3D models extracted from a video of a static scene. As in classical model-based video coding, the geometry, connectivity, and texture of the 3D models have to be transmitted, as well as the camera position for each frame in the original video. The proposed method is based on exploiting the interrelations existing between each type of information, instead of coding them independently, allowing a better prediction between the media and of the next information in the stream. Scalability is achieved through the use of wavelet-based representations for both texture and geometry of the models. A consistent connectivity is built for all 3D models extracted from the video sequence in order to get a consistent representation of the sequence possibly evolving in time. This allows a more compact representation and straightforward geometric morphing between successive time representations of the model. Furthermore this leads to consistent wavelet decomposition for 3D models in the stream. Targeted applications include distant visualization of the original video at very low bitrate and interactive navigation in the extracted 3D scene on heterogeneous terminals.

Wavelet methods for geometry encoding is a recently emerged superset of multiresolution analysis which has proven to be very efficient in terms of compression and adaptive transmission of three-dimensional (3-D) content. The decorrelating power and space/scale localization of wavelets enable efficient compression of arbitrary meshes as well as progressive and local reconstruction. Recent techniques based on zerotree compression have shown to be among the best lossy mesh compression methods, while remaining compatible with selective transmission of geometric data at various levels of detail. While some progressive reconstruction schemes have been proposed in the past, we show in this paper that this representation, recently proposed in the MPEG4 standard, can be efficiently used to perform real-time, view-dependent reconstruction of large meshes. The proposed system combines algorithms for local updates, cache management, and server/client dialog. The local details management is an improvement of progressive reconstructions built on top of hierarchical structures. It enables fast, homogeneous accommodation and suppression of wavelet coefficients at any level of subdivision, with time complexity independent of the size of the reconstructed mesh. The cache structure wisely exploits the hierarchical character of the received data, in order to avoid redundant information transmission. The whole system enables the client to have total control on the quality of navigation according to its storage and processing capabilities, whatever the size of the mesh.

We consider a new way to encode video sequences. The proposed method is based on second generation wavelets (SOW), a novel mathematic transform, successfully applied in 3D coding. This video coder is created by combining the wavelet theory and mesh geometry. With this and the SOW, we will be able to apply onto the video image, the mathematical transform to locate and to encode the error, results of the motion compensation process. This powerful signal processing theory allows a very well adapted coding of high frequencies. The main advantage is that these wavelets can be designed to fit exactly on singularities, shapes, textures, and edges. Hence, we can reduce redundancy and improve considerably coding efficiency over those peculiar settings. This new coder can be applied on all classical applications, such as very low bit rate transmissions, video streaming, vision conference and video on demand.

Wavelet methods for geometry encoding is a recently emerged superset of multiresolution analysis which has proven to be very efficient in term of compression and adaptive transmission of 3D content. The decorrelating power and space/scale localization of wavelets enable efficient compression of arbitrary meshes as well as progressive and local reconstruction. Recent techniques based on zerotree compression have shown to be among the best lossy mesh compression methods, while remaining compatible with selective transmission of geometric data at various level of detail. While some progressive reconstruction schemes have been proposed in the past, we show in this paper that this representation, recently proposed in the MPEG4 standard, can be efficiently used to perform real-time, view-dependent reconstruction of large meshes. The proposed system combines algorithms for local updates, cache management and server/client dialog. The local details management is an improvement of progressive reconstructions built on top of hierarchical structures. It enables fast, homogeneous accommodation and suppression of wavelet coefficients at any level of subdivision, with time complexity independent of the size of the reconstructed mesh. The cache structure wisely exploits the hierarchical character of the received data, in order to avoid redundant information transmission. The whole system enables the client to have total control on the quality of navigation according to its storage and processing capabilities, whatever the size of the mesh.

SS (subdivision surfaces) are a powerful modelling paradigm for truly hierarchical (instead of merely progressive) 3D surface (as opposed to mesh) coding. Two SS-based tools of the MPEG-4 animation framework extension allow us to derive a piecewise smooth surface from an initial control mesh: if the subdivision process is run in its predefined form, the initial mesh is simply smoothed; if 3D details are added to the positions of the new vertices appearing after each subdivision step, a particular target surface may be approximated with an increasing accuracy. In both cases, multiresolution editing/animation is possible.