This paper presents a new objective quality metric that was adapted to the complex characteristics of immersive video (IV) which is prone to errors caused by processing and compression of multiple input views and virtual view synthesis. The proposed metric, IV-PSNR, contains two techniques that allow for the evaluation of quality loss for typical immersive video distortions: corresponding pixel shift and global component difference. The performed experiments compared the proposal with 31 state-of-the-art quality metrics, showing their performance in the assessment of quality in immersive video coding and processing, and in other applications, using commonly used image quality assessment databases– TID2013 and CVIQ. As presented, IV-PSNR outperforms other metrics in immersive video applications and still can be efficiently used in the evaluation of different images and videos. Moreover, basing the metric on the calculation of PSNR allowed the computational complexity to remain low. Publicly available, efficient implementation of IV-PSNR software was provided by the authors of this paper and is used by ISO/IEC MPEG for evaluation and research on the upcoming MPEG Immersive video (MIV) coding standard.

In the paper we investigated the methods of obtaining high-resolution, high frame-rate virtual views based on low frame-rate cameras for applications in high-performance multiview systems. We demonstrated how to set up synchronization for multiview acquisition systems to record required data and then how to process the data to create virtual views at a higher frame rate, while preserving high resolution of the views. We analyzed various ways to combine time frame interpolation with an alternative side-view synthesis technique which allows us to create a required high frame-rate video of a virtual viewpoint. The results prove that the proposed methods are capable of delivering the expected high-quality, high-resolution and high frame-rate virtual views.

Delays in the delivery of immersive video to Head Mounted Devices (HMDs) are considered in the paper. The goal of this paper is to analyze the impact of video streaming latency on user experience with HMDs. The paper reports the results of the subjective quality assessment as a function of delay. The possible practical solutions are identified and, on this base, an experimental model of the considered systems is proposed. In order to properly test the influence of the streaming delay on the quality, the extensive subjective tests have been performed. Interesting conclusions have been drawn, showing that human acceptance of delay of translation of virtual point of viewing is much stronger than for delay of rotation of virtual viewing direction in immersive video. Finally, the influence of such observations on the system architectures is concluded.

The demonstration is aimed at presenting a simple free-viewpoint television system that is under development at Poznań University of Technology, Poznań, Poland. Video acquisition using sparsely distributed pairs of video cameras, the depth estimation from video, the virtual video rendering on a server feature the perspective FTV system. The original depth estimation and view synthesis algorithms ensure good quality of the virtual views generated during virtual walks around a scene. The system is developed for broadcasting sport (e.g. judo, karate, volleyball) and cultural (e.g. amateur or professional theater performances) events, as well as for interactive courses and manuals (medical, cosmetics, dancing, technical etc.).

The paper presents the concept of a practical free-viewpoint television system with purely optical depth estimation. The system consists of camera modules that contain pairs or triples of cameras together with the respective microphones. The camera modules can be sparsely located in arbitrary positions around a scene. Each camera module is equivalent to a video camera with a depth sensor and microphones. The hardware requirements, the video and audio processing algorithms and the preliminary experimental results are reported. In particular, for such systems, a compression technique is discussed that is more efficient than the new 3D-HEVC technology. A set of new test sequences obtained with the use of camera pairs are presented.

Rate-Distortion Optimized Quantization (RDOQ) is an encoding optimization technique that may be modified with no implications to the bitstream compliance with the standard. In the paper, we propose an RDOQ variant with exact cost estimation resulting in higher compression ratios as compared to HEVC Test Model. This improvement is obtained at the cost of the complexity increase that may be scaled by replacing the individual steps of exact estimation by the simplified ones from HEVC Test Model. In that way, a family of RDOQ variants is created. The performance of the exact variant from this paper defines a kind of the performance limit for RDOQ.

A computationally inexpensive watermark schema for MPEG-4 AVC/H.264 streams that use local mark strength adjustment minimizing visible artifacts is presented. The local mark strength adjustment mechanism and watermark embedding mechanism operate on residual data. This approach minimize the number of operations needed for video bitstream decoding and subsequent watermarked video data encoding. The experimental results included herein, described for real-world MPEG4 AVC/H.264 encoded HDTV video bitstreams prove its robustness to complex attacks, especially camcorder recording attack.

We propose a new coding technology for 3D video represented by multiple views and the respective depth maps. The proposed technology is demonstrated as an extension of the recently developed high efficiency video coding (HEVC). One base views are compressed into a standard bitstream (like in HEVC). The remaining views and the depth maps are compressed using new coding tools that mostly rely on view synthesis. In the decoder, those views and the depth maps are derived via synthesis in the 3D space from the decoded baseview and from data corresponding to small disoccluded regions. The shapes and locations of those disoccluded regions can be derived by the decoder without any side information transmitted. To achieve high compression efficiency, we propose several new tools such as depth-based motion prediction, joint high frequency layer coding, consistent depth representation, and nonlinear depth representation. The experiments show high compression efficiency of the proposed technology. The bitrate needed for transmission of two side views with depth maps is mostly less than 50% than that of the bitrate for a single-view video.

In this paper, we present the most efficient coding tools that are used in a new video compression technology for multiple views with the depth maps. This very well performing technology was designed and developed in response to MPEG Call for Proposals on 3D Video Coding Technology. The proposed technology exploits a tool that reduces the side views and the side depth maps to small disoccluded regions. This way only one central view and one depth map are coded in the HEVC syntax while the remaining views and the depth maps are synthesized in the decoder from the small disoccluded regions and from the central-view data. Therefore, the bitrate needed for a side view is mostly below 20% of the bitrate for single-view video.

The paper deals with multiview video coding using the new technology of High Efficiency Video Coding (HEVC). Implementation of multiview video coding in the framework of HEVC is described together with new specific tools proposed by the authors. Extensive experimental results are reported for compression performance comparison of MVC (ISO 14496-10), HEVC simulcast and two versions of proposed “multiview HEVC”. For “multiview HEVC” the results indicate significant bitrate reduction of about 50%, as compared to the state-of-the-art MVC technology standardized as a part of AVC (MPEG-4, H.264).

During the last two decades, a new technology generation of video compression was introduced about each 9 years. Each new compression-technology generation provides halving of necessary bitrates as compared to the last previous generation. This increasing single-view compression performance is related to increasing compression performance of multiview video coding. For multiview video with associated depth maps, additional significant bitrate reduction may be achieved. The paper reports the original compression technology that was designed and developed at Poznań University of Technology in response to MPEG Call for Proposals on 3D Video Coding Technology. The main idea of this technique is to predict very efficiently the side views and the depth maps from the base view.

The paper describes watermarking of compressed video bitstreams. This technique is proven to be very fast and robust against various attacks including camcorder recording attack. The watermark is embedded using indirect modifications of transform coefficients in selected P- and B-frames. In the watermarked bitstream there exist sequences of watermarked pictures interleaved with sequences of unmodified pictures. The unmodified pictures serve as a reference for the watermark detection. The experimental results described for real-world MPEG-4 AVC/H.264 encoded HDTV video bitstreams are included.