3D video is leading to the emergence of new technologies, such as virtual, augmented, and mixed realities, which find applications in several fields mainly healthcare, education, and industry. Even for the Internet of Things (IoT), the future lies in 3D vision and depth for machines such as autonomous cars, robots, and drones to have a deep perception like humans. Texture-dedicated compression methods are not efficient for depth maps due to features distinctions between depth and texture images. To tackle this issue, we propose a learning variable-rate depth map compression model with conditional quality-controllable autoencoder. Specifically, the encoder extracts deep features from depth maps through an advanced Convolutional Neural Network (CNN) model, trained using a combination of grayscale texture and depth images. This latter incorporates an initial layer of predefined wedgelet filters, succeeded by a VGG 19 pretrained model. To effectively differentiate between depth maps and grayscale textures within the training dataset, an image style classification technique using the learnt deep correlation Features is employed. Our end-to-end compression network ensures better performances over main candidate methods and depth-oriented 3D-HEVC compression standard.

With the emergence of depth sensors, extraction of depth maps features is becoming more and more solicited and prominent for several computer vision applications, such as gesture recognition, face recognition and segmentation. These applications can be more accurate thanks to the depth information that provides more precise separate foreground objects from background. In this paper, we propose an automatic depth maps features extraction model based on an optimized Convolutional Neural Network (CNN), trained on a mixture of depth maps and grayscale texture images. The CNN includes a first convolutional layer of pre-defined wedgelet filters, followed by a pre-trained VGG-19 neural network. Then, we opt for an image style classification based on Learnt Deep Correlation Features to capture features distinguishing depth maps from grayscale texture images of the training set. Experimental results demonstrate the potential effectiveness of the proposed wedgelet (We) and Learnt Deep Correlation Features (LDCF) based Network (WeLDCFNet) with a mean accuracy gain up to 32.77%, when compared to existing features extraction approaches. As our aim in this paper is depth maps features extraction and not the texture/depth classification itself, we propose, as a use case, to leverage the proposed WeLDCFNet for depth maps learned compression. If our model succeeds to extract depth features that make them distinguishable from texture, it would be useful to save the main compact depth information. Our WeLDCFNet based autoencoder, tailored to compression needs, shows competitive Rate/Distortion tradeoffs when compared to the latest depth maps compression standard.

Recently, deep learning has demonstrated impressive performance in image compression. Methods, that achieve and even outperform conventional codecs performances, are continually emerging. However, most of them need to train and deploy separate networks for rate adaptation. This is impractical and extensive in terms of memory cost and power consumption, especially for broad bitrate ranges. Further, methods that consider the semantic-important structure of the image are extremely sparse. This leads to non-optimized bit allocation for the eye-catching foreground details, that have to be preserved for the almost all computer vision applications. Towards this end, we establish an end-to-end multi-rate deep semantic image compression with quantized conditional autoencoder. It includes two neural networks for the semantic analysis and image compression, respectively. The semantic analysis network extracts the essential semantic regions of the input image, and calculates the Semantic-Important Structural SIMilarity (SI-SSIM) index for each of them. The compression network is then trained to optimize a multi-loss function based on SI-SSIM and conditioned on the activation bitwidths. Performances of our model are evaluated on the JPEG AI dataset for objective and perceptual quality metrics. Obtained results show that our method yields higher performances over JPEG, JPEG 2000 and HEVC intra baselines and competitive performances with VVC intra.

The conventional 2D metrics can be used for measuring the quality of depth maps, but none of them is considered to be efficient and is not accurate when used for evaluating 3D quality. In this paper, we propose a new full reference objective metric, called Sparse Representations-Mean Squared Error (SR-MSE), which efficiently evaluates the depth maps compression distortions. It adaptively models the reference and compressed depth maps in a mixed redundant transform domain dedicated to depth features. Then, it computes the mean squared error between the sparse coefficients issued from this modeling. As a benchmark of quality assessment, we perform a subjective evaluation test for depth maps compressed using the latest 3D High Efficiency Video Coding standard at various bitrates. We compare the subjective results with the proposed and conventional objective metrics. Experimental results demonstrate that the proposed SR-MSE, compared to the conventional image quality assessment metrics, yields the highest correlated scores to the subjective ones.

SHVC, the Scalable extension of the High Efficiency Video Coding standard (HEVC), combines large compression efficiency and high visual quality of different versions of a same video in a single bitstream. However, this comes at the cost of a high computational complexity. Many efforts aim to reduce this latter for texture images. In this paper, we aim at the same objective, but for depth maps that are characterized by areas of smoothly varying grey levels separated by sharp discontinuities at object boundaries. Typically, we propose a fast scalable coding scheme while exploiting depth maps specificities, depth information of SHVC Coding Units (CUs) as well as correlation between base and enhancement layers. Experimental results show that the proposed method significantly reduces the execution time of the SHVC encoder; while maintaining the quality of intermediate views synthesized from encoded depth maps.

The Dynamic Adaptive Streaming over HTTP (MPEG-DASH) ensures online videos display of good quality and without interruption. It provides an adequate streaming for each display device and network transmission. This can be very useful for the specific field of Massive Open Online Courses (MOOCs) where learners profit from an exceptional visual experience that improves their commitment level and eases the course assimilation. These MPEG-DASH assets can become more and more advantageous if a good choice of its parameters is made. Being a recent branch, the MPEG-DASH adaptive diffusion presents a research field where the efforts are still limited, even more for MOOC videos. Most of the work published in this sense focus on the Quality of Service (QoS) and the technical specifications of the network transmission. In this paper, we aim to consider the quality of the streamed content that directly impacts the learners quality of Experience (QoE). For this, we develop a content-aware dataset that includes several dashified MOOC videos. These latter are then exploited to study the most appropriate bitrates and segment durations for each type of MOOC videos.

Completely embedded in the 3D era, depth maps coding becomes a must in order to favour 3D admission to different fields of application, ranging from video games to medical imaging. This study presents a novel depth coding approach that, after a decimation step favouring the foreground, decomposes depth maps onto a set of sparse coefficients and redundant mixed discrete cosine and B‐splines atoms highly correlated to depth maps piece‐wise linear nature. Depth decomposition searches the best rate/distortion tradeoff through minimisation of an adaptive cost function, where its weight parameter is manipulated according to depth homogeneity. The bigger the parameter is, the more the sparsity is favoured at the expense of synthesis quality. Furthermore, handled distortion measure of the cost function quantifies the effect of depth maps coding on rendered views quality. The experiments show the relevance of the proposed method, able to obtain considerable tradeoffs between bitrate and synthesised views distortion.

Completely embedded in the 3D era, depth maps coding becomes a must in order to favor 3D admission to different fields of application, ranging from video games to medical imaging. This paper presents a novel depth coding approach that decomposes a decimated version of the original depth image on a sparse set of coefficients and mixed discrete cosine and B-splines atoms. The upstream decimation step reduces encoding bitrate without significant loss of virtual views quality. Depth decomposition is performed through minimization of an adaptive Rate/Distortion cost function, where we manipulate its weight parameter according to depth discontinuities. We then refine the choice of distortion metric in order to quantify the effect of depth maps coding on rendered views quality. Experiments show the relevance of the proposed method, able to obtain considerable tradeoffs between bitrate and synthesized views distortion.

Multiview video plus depth is the most popular 3D video representation that would support novel applications including free viewpoint television. These applications highly depend on high quality rendering of interpolated views which, as well, highly depends on the quality of decoded depth images. Therefore, a depth map coding that preserves perceptual quality, particularly on high frequency regions, is primary. In this paper, we propose a coding depth maps method that deals with emerging sparse signal decomposition technique. Depth images are approximated by a linear combination of few nonzero coefficients and dictionary atoms. Selected atoms are elementary signals based on a mixture of discrete cosine and B-splines of first degree. Sparse depth maps coding is tuned using a couple quality criterion such that depth discontinuities are preserved. The results investigated by objective evaluations over several depth maps imply that the proposed depth maps coding achieves better Rate-Distortion than JPEG and JPEG 2000. Subjective evaluation is also presented to stress the visual quality of interpolated views.