To effectively evaluate subjective visual quality in weakly-controlled environments, we propose an Adaptive Paired Comparison method based on particle filtering. As our approach requires each sample to be rated only once, the test time compared to regular paired comparison can be reduced. The method works with non-experts and improves reliability compared to MOS and DS-MOS methods.

In professional markets, there is a growing need to incorporate off-the-shelf consumer-level hardware to reduce costs and speed up development. Typically, these hardware components contain hardware-accelerated video encoders, which are limited to the more popular YUV4:2:0 profiles. For screen content coding (SCC) (e.g., desktop sharing, supervisory control and data acquisition applications, etc.), chroma subsampling can produce specific artifacts that are disturbing in high-quality professional applications. Current research on SCC focuses on developing specific screen content profiles and coding tools, which are not likely to be integrated in consumer electronics. In this paper, a method is explored to improve the chroma quality of SCC using YUV4:2:0 codecs. It is possible to recover parts of the chroma component based on the luma component and the subsampling filter that is used at the encoder device. A 1.63-dB peak signal-to-noise ratio-chroma improvement and 0.051 structural similarity-chroma improvement are measured on average for screen content sequences. The proposed method still provides improvements if compression is used.

High efficiency video coding (HEVC) shows a significant advance in compression efficiency and is considered to be the successor of H.264/AVC. To incorporate the HEVC standard into real-life network applications and a diversity of other applications, efficient bit rate adaptation (transrating) algorithms are required. A current problem of transrating for HEVC is the high computational complexity associated with the encoder part of such a cascaded pixel domain transcoder. This paper focuses on deriving an optimal strategy for reducing the transcoding complexity with a complexity-scalable scheme. We propose different transcoding techniques which are able to reduce the transcoding complexity in both CU and PU optimization levels. At the CU level, CUs can be evaluated in top-to-bottom or bottom-to-top flows, in which the coding information of the input video stream is utilized to reduce the number of evaluations or to early terminate certain evaluations. At the PU level, the PU candidates are adaptively selected based on the probability of PU sizes and the co-located input PU partitioning. Moreover, with the use of different proposed methods, a complexity-scalable transrating scheme can be achieved. Furthermore, the transcoding complexity can be effectively controlled by the machine learning based approach. Simulations show that the proposed techniques provide a superior transcoding performance compared to the state-of-the-art related works. Additionally, the proposed methods can achieve a range of trade-offs between transrating complexity and coding performance. From the proposed schemes, the fastest approach is able to reduce the complexity by 82% while keeping the bitrate loss below 3%.

The visualization of (ultra) high-resolution compositions created from multiple input bitstreams requires several decoders at the receiving device. Therefore, not all devices can properly display such compositions. To address this problem, the input streams are decoded, merged into a single video, and re-encoded by a transcoder in the network. However, this approach requires a computationally complex re-encoding step. To reduce this complexity, information from the input bit-streams can be reused during transcoding. In HEVC, simply reusing the original encoding information is not compression efficient if the inserted content is not aligned with the grid of coded blocks. In this paper, we applied feature selection based on a decision tree, which was used in a fast HEVC transcoding model for misaligned content. The performance varies depending on the shift and average transform size in the original sequence, resulting in complexity reductions of up to 76%.

Transrating is a useful tool for adapting the bitrate of a video stream. Reducing the complexity of transrating is extremely important, especially for high efficiency video coding (HEVC) based transrating where the encoder is very time consuming. TZSearch is currently the default integer motion estimation (ME) algorithm in the HEVC reference encoder thanks to its excellent performance in reducing the complexity of ME. However, this algorithm is characterized by a fixed search area and search pattern, which can be considered sub optimal for transrating. In this paper, we improve the performance of HEVC transrating by optimizing this TZSearch algorithm. Utilizing the correlation between input and output motion vectors, we propose a fast search scheme including three steps. First, the initial search point is selected. Then, using the rate-distortion cost of this starting point, the search size is determined according to an online-trained Bayes decision rule. Finally, two proposed search algorithms are described for refining the starting point. Experimental results show that our proposed TZSearch scheme can reduce the complexity of transrating while improving the coding performance in terms of bit rate saving.

The capabilities of off-the-shelf consumer electronics are continuously increasing, making them interesting candidates to integrate into industrial visualization systems for reducing the cost of these systems or speed up development. However, in the context of video compression, hardware acceleration on consumer electronics is typically only provided for more common, consumer oriented YUV 4:2:0 profiles. Meanwhile, industrial applications often require YUV 4:4:4 formats due to synthetic visual data. Upscaling YUV 4:2:0 to YUV 4:4:4 using conventional filters results in visual artifacts, specifically for synthetic content. To improve the quality of the chroma components, this paper proposes to extend the well-known guided image filter through content-adaptive selection of the filter radius, and analyze different strategies for limiting complexity. This results in a 3.5 dB improvement of the chroma PSNR compared to conventional filters.

In an IPTV environment, different techniques exist to provide faster random access or equivalently, a faster channel switching experience, but none of them provide backward compatibility and limited overhead at the same time. In this paper, a technique is proposed to increase the random access frequency for the High Efficiency Video Coding (HEVC) standard by using a single-loop scalable version of it. This is achieved by encoding an enhancement layer with a higher frequency of random access points compared to the base layer. With this scalable configuration, a backward compatible base stream remains present. The proposed technique requires 12.4% less bitrate compared to sending the fast and slow switching streams in a non-scalable way. Compared to the almost standardized multi-loop scalable extension of HEVC, 5.5% less bitrate is needed on the core of the IPTV network and 12.5% less bitrate on the access network during steady state conditions. Moreover, the multi-loop scalable extension does not provide backward compatibility with legacy HEVC decoders.

One of the drawbacks of current 3DTV systems are the glasses required to separate the 3D views. This problem can be solved by using autostereoscopic 3D displays, which generates a 3D scene from different viewpoints. These viewpoints are synthesized in the display based on depth information of the scene. However, currently no depth information is available for stereoscopic video. Therefore, depth estimation based on the stereoscopic input is required. This depth estimation should be low-complex and real-time for consumer electronic devices. Due to this low-complexity requirement, the quality of such depth maps will be limited. In this paper, we propose a guided depth filter, which filters the estimated depth map by using the downsampled chroma signal of the corresponding texture view as a guide. This allows to refine the borders of objects which reduces visible inpainting artifacts caused by the view synthesis. The proposed algorithm has been evaluated by expert viewing to show that it is independent of the depth map estimation and view synthesis of the autostereoscopic display.

To incorporate the newly developed High Efficiency Video Coding (HEVC) standard in real-life network applications, efficient transrating algorithms are required. We propose a fast transrating scheme, based on the early prediction of the partition split-flags in P pictures. Using machine learning techniques, the correlation between co-located partitions at different quantizations is investigated. This results in a model which predicts the split-flag and gives the associated prediction accuracy so that the splitting process in the transcoder is optimized. At each partition depth, the model indicates whether the full rate-distortion cost evaluations should be performed at the current depth, or if the partition can be split immediately. Experimental results show that the proposed transcoder reduces the complexity of the transrating process by 76.04%, while maintaining the coding efficiency of a cascaded decoder-encoder.

When middlebox devices should be able to adapt an encrypted video stream in the network without having the decryption key, format-compliant partial encryption schemes should be applied. In this paper, we propose such encryption schemes for the recently standardized High Efficiency Video Coding (HEVC) standard. By encrypting specific syntax elements like the sign of the residual information, the sign of the motion vector (MV) difference, the MV prediction index, and the MV reference index, format compliance and the possibility for adaptation are offered. Scrambling performance gradually increases when shifting from encrypting the motion information to encrypting the residual sign and finally to the combination thereof. Applying all these techniques has a negligible impact on the compression efficiency.

Nowadays, networks and terminals with diverse characteristics of bandwidth and capabilities coexist. To ensure a good quality of experience, this diverse environment demands adaptability of the video stream. In general, video contents are compressed to save storage capacity and to reduce the bandwidth required for its transmission. Therefore, if these compressed video streams were compressed using scalable video coding schemes, they would be able to adapt to those heterogeneous networks and a wide range of terminals. Since the majority of the multimedia contents are compressed using H.264/AVC, they cannot benefit from that scalability. This paper proposes a technique to convert an H.264/AVC bitstream without scalability to a scalable bitstream with temporal scalability in Main Profile by accelerating the mode decision task of the SVC encoding stage using Machine Learning tools. The results show that when our technique is applied, the complexity is reduced by 87% while maintaining coding efficiency.

Scalable video coding (SVC) allows encoded bitstreams to be adapted. However, most bitstreams do not incorporate this scalability so bitstreams have to be adapted multiple times to accommodate for varying network conditions or end-user devices. Each adaptation incorporates an additional loss of quality due to transcoding. To overcome this issue, we propose a single transcoding step from H.264/AVC to SVC. Doing so, the resulting bitstream can be freely adapted without any additional quality reduction. Open-loop transcoding architectures can be used for H.264/AVC-to-SVC transcoding with a low complexity, although these architectures suffer from drift artifacts. Closed-loop transcoding, on the other hand, requires a higher complexity. To overcome the drawbacks of both systems, we propose combining both techniques.

In the current development of HEVC, compression performance improved significantly compared to H.264/AVC for both inter pictures and intra pictures. With intra compression, the main reason for this improvement is the large in crease in intra prediction directions (up to 34). The downside of having a larger number of modes is that they increase the signaling overhead in the bitstream. In this paper, a low complexity intra mode prediction algorithm is proposed which improves the mode prediction accuracy. This is achieved by exploiting the correlation between the prediction directions of the neighboring prediction units and that of the encoded prediction unit. As a result, more efficient intra mode signaling can be achieved with minimal impact on encoder and decoder complexity. On average, 0.33% bitrate improvement is obtained by employing the proposed algorithm. For sequences that are encoded with a high number of directional intra modes, around 1% bitrate improvement is measured.

Scalable Video Coding (SVC) is a recent extension of the ISO/ITU Advanced Video Coding (H.264/AVC) standard, which allows adapting the bitstream easily by dropping some parts of it named as layers. This adaptation makes possible that the same bitstream meets the requirements for reliable delivery of video to diverse clients over heterogeneous networks using temporal, spatial or SNR scalability, combined or separately. Since the SVC design requires scalability to be provided at the encoder side, the existing contents cannot benefit of it. Efficient techniques for converting contents without scalability to a scalable format are desirable. In this paper, an approach for temporal scalability transcoding from H.264/AVC to SVC is presented and the impact of the GOP size is analyzed. Independently of the GOP size chosen, around a 60% of time saving is achieved while maintaining the coding efficiency. Moreover, this technique could be used to transform an H.264/AVC bitstream without temporal scalability to another H.264/AVC bitstream with hierarchical structures that provides temporal scalability.