Along with the rapid increase in the availability of high-quality video formats such as high definition (HD), ultra HD, and high dynamic range, a huge demand for data rates during their transmission has become inevitable. Consequently, the role of video compression techniques has become crucially important in the process of mitigating the data rate requirements. Even though the latest video codec high efficiency video coding (HEVC) has succeeded in significantly reducing the data rate compared to its immediate predecessor H.264/advanced video coding, the HEVC coded videos in the meantime have become even more vulnerable to network impairments. Therefore, it is equally important to assess the consumers' perceived quality degradation prior to transmitting HEVC coded videos over an error-prone network, and to include error resilient features so as to minimize the adverse effects of those impairments. To this end, this paper proposes a probabilistic model which accurately predicts the overall distortion of the decoded video at the encoder followed by an accurate QP-λ relationship which can be used in the rate-distortion optimization (RDO) process. During the derivation process of the probabilistic model, the impacts from the motion vectors, the pixels in the reference frames, and the clipping operations are accounted, and consequently, the model is capable of minimizing the prediction error to as low as 3.11%, whereas the state-of-the-art methods cannot reach below 20.08%, under identical conditions. Furthermore, the enhanced RDO process has resulted in 21.41%-43.59% improvement in the BD-rate compared to the state-of-the-art error resilient algorithms.

Determining the best partitioning structure of a coding tree unit is one of the most time-consuming operations in High Efficiency Video Coding (HEVC) encoding. Specifically, it is the evaluation of the quadtree hierarchy using the rate-distortion (RD) optimization that has the most significant impact on the encoding time, especially in the cases of high definition (HD) and ultra HD videos. In order to expedite the encoding for low-delay applications, this paper proposes a coding unit (CU) size selection and encoding algorithm for inter prediction in the HEVC. To this end, it describes: 1) two CU classification models based on Inter N × N mode motion features and RD cost thresholds to predict the CU split decision; 2) an online training scheme for dynamic content adaptation; 3) a motion vector reuse mechanism to expedite the motion estimation process; and 4) finally introduces a computational complexity to coding efficiency tradeoff process to enable flexible control of the algorithm. The experimental results reveal that the proposed algorithm achieves a consistent average encoding time performance ranging from 55%-58% and 57%-61% with average Bjøntegaard delta bit rate increases of 1.93%-2.26% and 2.14%-2.33% compared with the HEVC 16.0 reference software for the low delay P and low delay B configurations, respectively, across a wide range of content types and bit rates.

In the absence of a commercial High Dynamic Range (HDR) distribution pipeline, two-layer backward-compatible HDR video coding is a viable solution for the imminent transition from Low Dynamic Range (LDR) to HDR content transmission. However, the performance of a two-layer coding solution is governed by the extension layer coding performance. In this paper, we propose an improved two-layer backward-compatible HDR video coding solution based on an adaptive residual mapping for the extension layer, keeping in view the performance of High Efficiency Video Coding (HEVC) being used to code this information. The proposed solution outperforms the reference method achieving averaged PU-PSNR improvements of up to 5.05 dB. The proposed method also shows potential of achieving the same HDR quality as the single layer coding solution with a minimum bitrate overhead and acceptable LDR quality in the base layer.

The rising demand for media consumption via mobile devices and the emergence of complex video coding algorithms present an additional challenge for the energy management algorithms in resource-constrained consumer electronic devices. Thus, generating energy-optimized video bit streams will positively contribute towards overcoming this challenge. This paper introduces a novel energy model for intra-frame decoding of HEVC encoded video that predicts the decoding energy of a coding unit. Thereafter, a novel energy-rate-distortion optimized coding mode selection algorithm is proposed to generate energy-efficient bit streams using the proposed energy model within the encoder. The proposed energy model is shown to predict the decoding energy of a coding unit with an average error less than 2%. Moreover, the proposed coding mode selection algorithm achieves an average 10.8% reduction in the energy consumed at the decoder with a −0.25 dB impact to the Bj⊘ntegaard Delta-Peak Signal-to-Noise Ratio.

Social interaction of groups of users, amongst themselves and with the media content itself, is increasingly becoming popular due to the advancements in the Internet access technologies. However, multimedia resource provisioning for dispersed user groups poses a challenge and demands innovative technologies. This paper proposes a novel approach based on Particle Swarm Optimization (PSO) to optimally allocate computational and networking resources to a group of interactive users, such that the group Quality-of-Service (QoS) is maximized. We evaluate the performance of the proposed improved PSO method with respect to the state-of-the-art greedy resource allocation mechanisms and related PSO approaches. The ability to find a feasible solution (i.e., the serving probability) and the accuracy of such solutions are compared for different network topologies. The proposed method demonstrates reduced computational complexity, an up to 40% increase in the serving probability compared to the greedy methods, and up to 60 times faster convergence compared to the basic PSO approach. Overall, the comparable QoS level to the optimal solution suggests that the proposed solution efficiently allocates the resources available in the network.

High Dynamic Range (HDR) is a key technology envisioned to provide the end-users a sense of “being there”. With the advent of HDR displays in the near future, it is not clear if the existing content distribution environment will be capable of supporting HDR. Although it is generally argued that single-layer HDR coding solutions are superior to two-layer backward-compatible techniques, these coding strategies have not been fully investigated in terms of their rate distortion trade-off. In this paper, three main HDR coding techniques are implemented to analyze their performance and to evaluate the potential of HDR content distribution, while minimizing the impact on existing infrastructure, i.e., in terms of bitrate and HDR quality trade-off. The results reveal that backward-compatible solutions can outperform the single-layer approach achieving PSNR_DE improvements of up to 3.05dB on average. The HDR-VDP-2 quality indexes show similar improvements further confirming the superior performance of the backward-compatible solutions.

High spectral efficiency is at the core of any effort made to achieve the data rates and capacity requirements demanded by next generation mobile technologies. Adaptive Modulation and Coding (AMC) is considered a powerful tool for the efficient usage of spectrum and for the provision of channel adaptive error resilience. In this paper, a novel AMC based transmission scheme is introduced for Long Term Evolution-Advanced (LTE-A) networks to impose adaptive error resilience on High Efficiency Video Coding (HEVC) based video data at the physical layer. A model is proposed to rank REs at the LTE-A transmitter based on channel information. Data are allocated onto ranked REs to maximise the efficiency of the users' allocated bandwidth while maintaining video quality. This is achieved by imposing a stronger Modulation and Coding Scheme (MCS) for data that have a high probability of being distorted, thus spanning the entire allocated bandwidth. Simulation results show that the proposed Unequal Error Protection (UEP) strategy helps to minimise the LTE-A transmitter power by up to 8dB with guaranteed video quality. In effect, this is a reduction of approximately 84% of the transmit power.

Determining the best partitioning structure for a given Coding Tree Unit (CTU) is one of the most time consuming operations within the HEVC encoder. The brute force search through quad tree hierarchy has a significant impact on the encoding time especially on high definition (HD) videos. This paper presents a fast coding unit size decision-taking algorithm for inter prediction in HEVC. The proposed algorithm uses a motion homogeneity based classification approach utilizing RD cost as a feature vector. Simulation results show that the proposed algorithm achieves an average of 73.25% encoding time efficiency improvement with similar rate distortion performance compared to HEVC HM12.0 reference software.

Determining the best partitioning structure for a given Coding Tree Unit (CTU) is one of the most time consuming operations within the HEVC encoder. The brute force search through quadtree hierarchy has a significant impact on the encoding time of high definition (HD) videos. This paper presents a fast coding unit size decision-taking algorithm for intra prediction in HEVC. The proposed algorithm utilizes a low complex texture analysis technique based on the local range property of a pixel in a given neighborhood. Simulation results show that the proposed algorithm achieves an average of 72.24% encoding time efficiency improvement with similar rate distortion performance compared to HEVC reference software HM12.0 for HD videos.

This paper proposes content and network-aware redundancy allocation algorithms for channel coding and network coding to optimally deliver data and video multicast services over error prone wireless mesh networks. Each network node allocates redundancies for channel coding and network coding taking in to account the content properties, channel bandwidth and channel status to improve the end-to-end performance of data and video multicast applications. For data multicast applications, redundancies are allocated at each network node in such a way that the total amount of redundant bits transmitted is minimised. As for video multicast applications, redundancies are allocated considering the priority of video packets such that the probability of delivering high priority video packets is increased. This not only ensures the continuous playback of a video but also increases the received video quality. Simulation results for bandwidth sensitive data multicast applications exhibit up to 10× reduction of the required amount of redundant bits compared to reference schemes to achieve a 100% packet delivery ratio. Similarly, for delay sensitive video multicast applications, simulation results exhibit up to 3.5dB PSNR gains in the received video quality.

As the limits of video compression and usable wireless radio resources are exhausted, providing increased protection to critical data is regarded as a way forward to increase the effective capacity for delivering video data. This paper explores the provisioning of selective protection in the physical layer to critical video data and evaluates its effectiveness when transmitted through a wireless multipath fading channel. In this paper, the transmission of HEVC encoded video through an LTE-A wireless channel is considered. HEVC encoded video data is ranked based on how often each area of the picture is referenced by subsequent frames within a GOP in the sequence. The critical video data is allotted to the most robust OFDM resource blocks (RBs), which are the radio resources in the time-frequency domain of the LTE-A physical layer, to provide superior protection. The RBs are ranked based on a prediction for their robustness against noise. Simulation results show that the proposed content aware resource allocation scheme helps to improve the objective video quality up to 37dB at lower channel SNR levels when compared against the reference system, which treats video data uniformly. Alternatively, with the proposed technique the transmitted signal power can be lowered by 30% without sacrificing video quality at the receiver.

Network Coding is a promising approach to increase network throughput and robustness to facilitate high volume traffic. Performing network coding in dynamic network structures requires transmitting coding coefficients for information sinks to decode network coded packets. Compared to the packet sizes used in practical networks, the size of coefficient vectors can be significant. This paper exploits the properties of small and medium sized networks and proposes a novel approach to minimise the coefficient vector size of network coded packets. Simulation results exhibit better compression of coefficient vectors over existing algorithms for small and medium sized networks.

The quality assessment of impaired stereoscopic video is a key element in designing and deploying advanced immersive media distribution platforms. A widely accepted quality metric to measure impairments of stereoscopic video is, however, still to be developed. As a step toward finding a solution to this problem, this paper proposes a full reference stereoscopic video quality metric to measure the perceptual quality of compressed stereoscopic video. A comprehensive set of subjective experiments is performed with 14 different stereoscopic video sequences, which are encoded using both the H.264 and high efficiency video coding compliant video codecs, to develop a subjective test results database of 116 test stimuli. The subjective results are analyzed using statistical techniques to uncover different patterns of subjective scoring for symmetrically and asymmetrically encoded stereoscopic video. The subjective result database is subsequently used for training and validating a simple but effective stereoscopic video quality metric considering heuristics of binocular vision. The proposed metric performs significantly better than state-of-the-art stereoscopic image and video quality metrics in predicting the subjective scores. The proposed metric and the subjective result database will be made publicly available, and it is expected that the proposed metric and the subjective assessments will have important uses in advanced 3D media delivery systems.

With the increasing popularity of online video streaming and broadcasting applications, video transmission has become the most bandwidth consumed application in the Internet. Unless resource utilisation is further optimised, existing networks would not be able to accommodate the forecasted demand for high volume video traffic. Random Linear Network Coding (RLNC) is a promising approach to increase the network throughput and robustness in order to facilitate high volume traffic. However, the performance gain obtained using RLNC decreases when the network packet loss rate increases. In response, this paper proposes to incorporate RLNC with a combined interleaving and a coding technique to improve the effectiveness of generating redundant packet combinations over classical RLNC to enhance the robustness against packet losses. Simulation results illustrate that the proposed method improves the probability of recovering lost data over the classical RLNC and the video transmission simulation results exhibit up to 5dB increase in PSNR and up to 14% increase in VQM utilising the same network resources at the same network conditions.

Three different advanced adaptation techniques for improving the video perception of users are proposed in this paper. The proposed techniques exploit different adaptation decision-taking and adaptation approaches to adapt particular core parameters while considering diverse contextual information and constraints to achieve improved video perception of users. The first proposed technique employs a utility-based adaptation approach to perform adaptation operations on spatial resolution, frame rate, and quality scalability parameters according to the content-related contextual information (i.e., motion activity and structural feature) while fulfilling network bandwidth and terminal display size constraints. Using this technique, video contents can be adapted with the scalability parameters best fitting users' and contextual constraints' needs to achieve improved video perception. The second technique relies on prioritizing key frame, non-key frame, and temporal layer parameter-related network abstraction layer units to adapt video contents to satisfy network bandwidth constraint. The rate-distortion performances of adapted video contents can be improved by utilizing this technique in adaptation operations both in terms of bit rate of adapted video contents and video perception of users. The third technique is based on adapting the bit rate of 3-D video contents according to the changes in ambient illumination of the viewing environment. The adaptation results evaluated by either subjective or objective quality assessment techniques prove that all of the proposed techniques are efficient to improve the video perception of users.

Three-dimensional (3-D) displays provide a more realistic experience of entertainment by providing its viewers an added sensation of depth by artificially exploiting light rays to stimulate certain depth cues in the human visual system, especially binocular stereopsis. Due to its close relationship with human visual perception, mass market deployment of 3-D displays will be significantly dependant upon addressing the related perceptual factors such as visual comfort. In order to address the perceptual factors, it is very important to understand how humans experience depth on 3-D displays and how sensitive they are for different depth cues. In this paper, the sensitivity of humans for different depth cues is analyzed as applicable to 3-D viewing on stereoscopic displays. Mathematical models are derived to explain the just noticeable difference in depth (JNDD) for three different depth cues, namely binocular disparity, retinal blur, and relative size. Extensive subjective assessments are performed on a stereoscopic display with passive polarized glasses and on an auto-stereoscopic display to validate the mathematical models for JNDD. It is expected that the proposed models will have important use cases in 3-D display designing as well as 3-D content production.

Compression of Depth maps that are used in 3D video systems based on Depth Image Based Rendering (DIBR) poses a new challenge in video coding, since it is not a sequence of images for final viewing by end users rather an aid for rendering. Therefore, compressing depth maps using existing video coding techniques yields unacceptable distortions while rendering virtual views. In this paper we propose a novel mode selection method for offline compression of depth maps by selecting modes collaboratively considering an entire row of macroblocks together. For selecting these modes while encoding, we propose a novel distortion criteria that incorporates rendering distortions instead of distortion of depth map itself. A genetic algorithm based optimization technique is used for the mode selection. The simulation results suggest that the proposed technique can improve the PSNR up to 1.6dB in the rendered stereoscopic views in comparison to the block wise mode selection method based on Lagrange Optimization and the distortion of the depth map itself.

In this letter, a new bit rate adaptation decision taking technique is proposed based on the observations that ambient illumination context has an effect on the perceived video quality. Motion activity and structural complexity characteristics of a video content are utilized as the content-related contexts in the proposed technique. Experimental results demonstrate that a significant amount of bit rate can be saved while maintaining the perceptual quality same by adapting the video content according to different ambient illumination conditions using this technique. Subjective assessment results show that the proposed adaptation technique is capable of exploiting the changes in ambient illumination level for bit rate adaptation without sacrificing the perceived visual quality.

This paper proposes a framework for a virtual classroom application based on a Virtual Collaboration System (VCS), which is being developed under the VISNET II Network of Excellence (NoE)1, and discusses adaptation technologies that enable seamless access to classroom sessions while intellectual property and digital rights are managed. The proposed virtual classroom framework enables academic institutions to conduct their collaborative lecture series, to which registered students will be able to attend remotely and interactively over the Internet. Furthermore, the general public may also follow the classroom sessions under certain restrictions imposed by the participating institutions. In order to facilitate seamless access to a heterogeneous audience that is composed of users with various preferences and privileges accessing the classroom sessions over different network infrastructures and using terminal devices with diverse capabilities, context-aware content adaptation is required to meet constraints imposed by the usage context and enhance the quality of the user experience. Thus, this paper describes the concepts and functionalities of a context-aware content adaptation platform that suits the requirements of such multimedia application scenarios. This platform is able to consume low-level contextual information to infer higher-level contexts, and thus decide the need for and type of adaptation operations to be performed upon the content. In this way, it is aimed to meet usage constraints while also satisfying restrictions imposed by the Digital Rights Management (DRM) to govern the use of protected content.

Distributed video coding (DVC) is one of the fastest growing coding techniques in the current signal processing world. It has its outstanding low complex encoder which is essential for some applications in the consumer market. Driven by the above motive, in this paper we propose a novel side information refinement technique for Wyner-Ziv video frames in DVC. When a bit plane is decoded, it is used to replace the corresponding bit plane in the side information and used for motion estimation and compensation to up date the side information. Then the information for the next bit plane is extracted from the updated side information and is sent to the turbo decoder as in a conventional DVC decoder. This process is repeated until all bit planes are extracted. This technique can be employed with almost all the side information generation techniques from simplest conventional pixel interpolation to advanced techniques such as bi-directional motion estimation and spatial motion smoothing (BiMESS). Simulation results show that over 1.4 dB PSNR gain can be obtained with the proposed algorithm over the BiMESS smoothing algorithm at the same bit rate.

Owing to the provision of a more natural and realistic reconstruction of a scene in the form of two views presented to the left and right eyes, stereoscopic imaging/video technology offers a more attractive and effective solution for imaging/video production. However due to the fact that more information has to be transmitted to reproduce the stereoscopic image pair, the transmission bit rate might be considerably increased if the available redundancies are not properly exploited. This paper presents a novel motion and disparity vector prediction technique for the H.264 based stereoscopic video codec that can be used for future consumer video applications such as 3D TV and 3D mobile communication in order to generate the real 3D perception in the human psycho-visual system. The proposed technique predicts both motion and disparity vectors more intelligently using the previously coded motion and disparity vectors of the surrounding macroblocks. Results show that the proposed algorithm can reduce the average bit budget by about 12% compared to the conventional motion and disparity prediction algorithm employed by the H.264 (Wiegand, 2003) based stereoscopic codec (Adikari, 2005). Furthermore, results suggest that the objective quality of the sequences is also improved.

One of the significant problems in video compression schemes is the high fluctuation in output data rate over the video sequence. These compression schemes, in general, utilize a rate control algorithm in order to maintain the output data rate at a constant level regardless of the properties of the video sequence and differences in compression ratios of different picture types. In this paper, we propose a novel rate control algorithm with the particle swarm optimization (PSO) (J. Kennedy and R.C. Ebehart, 1948) technique for offline MPEG-1/2 codecs. We model the rate control problem as an optimization problem and use the PSO to obtain a better optimal solution. Experimental results show that the proposed method can improve the average picture signal to noise ratio (PSNR) by more than 2 dB.