In this paper, we discuss a region-based perceptual quality-regulable H.264 video encoder system that we developed. The ability to adjust the quality of specific regions of a source video to a predefined level of quality is an essential technique for region-based video applications. We use the structural similarity index as the quality metric for distortion-quantization modeling and develop a bit allocation and rate control scheme for enhancing regional perceptual quality. Exploiting the relationship between the reconstructed macroblock and the best predicted macroblock from mode decision, a novel quantization parameter prediction method is built and used to achieve the target video quality of the processed macroblock. Experimental results show that the system model has only 0.013 quality error in average. Moreover, the proposed region-based rate control system can encode video well under a bitrate constraint with a 0.1% bitrate error in average. For the situation of the low bitrate constraint, the proposed system can encode video with a 0.5% bit error rate in average and enhance the quality of the target regions.

In this work, a perceptual quality-regulable H.264 video encoder system has been developed. Exploiting the relationship between the reconstructed macro block and its best predicted macro block from mode decision, a novel quantization parameter prediction method is built and used to regulate the video quality according to a target perceptual quality. An automatic quality refinement scheme is also developed to achieve a better usage of bit budget. Moreover, with the aid of salient object detection, we further improve the quality on where human might focus on. The proposed algorithm achieves better bit allocation for video coding system by changing quantization parameters at macro block level. Compared to JM reference software with macro block layer rate control, the proposed algorithm achieves better and more stable quality with higher average SSIM index and smaller SSIM variation.

Image quality assessment (IQA) is a crucial feature of many image processing algorithms. The state-of-the-art IQA index, the structural similarity (SSIM) index, has been able to accurately predict image quality by assuming that the human visual system (HVS) separates structural information from non-structural information in a scene. However, the precision of SSIM is relatively lacking when used to access blurred images. This paper proposes a novel metric of image quality assessment, the JND-SSIM, which adopts the just-noticeable difference (JND) algorithm to differentiate between plain, edge, and texture blocks and obtain a visibility threshold map. Based on varying block transition types between the reference and distorted image, SSIM values are assigned respective weights and scaled down by visibility threshold map. We then test our algorithm on the LIVE and TID Image Quality Database, thereby demonstrating that our improved IQA index is much closer to human opinion.

In this paper, a perceptual quality regulable H.264 video encoder system has been developed. We use structure similarity index as the quality metric for distortion-quantization modeling and develop a bit allocation and rate control scheme for enhancing regional perceptual quality. Exploiting the relationship between the reconstructed macroblock and its best predicted macroblock from mode decision, a novel quantization parameter prediction method is built and used to regulate the video quality of the processing macroblock according to a target perceptual quality. Experimental results show that the model can achieve high accurate. Compared to JM reference software with macroblock layer rate control, the proposed encoding system can effectively enhance perceptual quality for target video regions.

Error concealment techniques can improve subjective video quality in a decoder when the video bitstream is corrupted during transmission. In this paper, to achieve perceptually pleasant results, an image inpainting technique in which structure information generated from edge information is adopted as the spatial error concealment method. In addition, a modified boundary matching algorithm for temporal error concealment is proposed for temporal frames. To maintain low hardware costs as regards the error concealment engine, the processing iteration number of each macroblock is limited to four based on the proposed inpainting algorithm. Block-based pipeline scheduling is also proposed to reduce the number of processing cycles and the on-chip memory size. Moreover, a cache-based data reuse scheme is developed to reduce the processing cycles and external bandwidth. Moreover, the two concealment modes share the same computational core to reduce hardware costs. A prototype chip is implemented by using the UMC 90 nm process. The total gate count is approximately 121 k at 200 MHz. The maximum processing capability can support 244.8 k macroblocks per second or 1920 × 1080 4:2:0 30 Hz video. The core size is 1.30 × 1.30 mm2. The average power dissipation is 131.4 mW at 200 MHz. Compared to other error concealment methods, the proposed design can achieve better perceptual quality at an acceptable additional hardware cost.

In image and video coding field, an effective compression algorithm should remove not only the spatial, temporal, and statistical redundancy but also the perceptual redundancy information from the pictures. Many perceptual models are presented in the literature to cooperate with video coding system to obtain significant bit rate reduction without perceptual distortion. One of the critical issues for those perceptual models is their high computational complexity to apply to real-time applications. To alleviate this problem, this paper aims at hardware architecture design of perception engine for video coding applications. The adopted perceptual models include the structural similarity model, visual attention models, and just-noticeable-distortion model, and contrast sensitivity function. Moreover, those models are further developed and modified to be suitable for hardware implementation. Macroblock-based processing with data reuse scheme is used to save the system bandwidth. The architecture of parallel processing for each visual model with sharing the on-chip memory and buffers is developed to reduce the chip area. Subjective experiment results show that the adopted model achieves about 7%-41% bit-rate saving in the QP range of 24-36 without visual quality degradation. For the hardware implementation of the perception engine, the chip is taped out using 0.18 m technology. The chip size is about 3.3 3.3 mm , and the power consumption is 83.9 mW. The processing capability is HDTV720p.

In order to increase coding efficiency, a perception-aware H.264/AVC encoder is proposed in this paper. With a different perception models for intra-frames and inter-frames, the initial quantization parameter (QP) is perceptually adjusted to remove perceptual redundancy. Moreover, the associated hardware architectures of the whole encoder and the perception analysis engine are also proposed with hardware sharing and pipelining design techniques. Experimental results show that our encoder achieves about 6-27% bit-rate saving in the QP range of 28-36. Furthermore, the hardware cost of the perception analysis engine is 120K gates, which is acceptable to be integrated into an H.264/AVC encoder chip.

This paper presents an efficient error concealment algorithm for video bitstream over error-prone channel suffering from damage. Moreover, hardware architecture design and chip implementation of the proposed error concealment algorithm are also presented. For spatial error concealment, a mode selection algorithm considering the reuse of intra mode information embedded in bitstream is developed for the adaptation of bilinear and directional interpolation. It suffers only 0.08 dB video quality drop in average but the speedup measured on a general purpose processor is up to 40 times compared with the conventional methods. It is also more suitable for low cost hardware design. For temporal error concealment, the decoded motion vectors of the neighboring blocks of the corrupted macroblock are reused to provide hints to estimate the motion vector of the corrupted macroblock. Moreover, for real-time applications, a data and computational results reuse scheme of motion vector estimation is proposed and 96% computation and memory bandwidth can be reduced compared with the conventional methods with 0.18 dB quality drop in average. With the UMC 90 nm 1P9M process, the proposed error concealment engine can process HDTV1080P 30 frames per second video data and the power consumption is 15.77 mW at 125 MHz operation frequency.

A bio-inspired perceptual video coding algorithm which can reduce the bit rate of the encoded bit stream while maintaining equal perceptual (visual) quality is proposed in this paper. Instead of applying the complicated bio-inspired perceptual model, we propose an algorithm which automatically analyzes the spatial and temporal information in the video sequence, and achieves better bit allocation for video coding systems by changing the quantization parameter (QP) in macroblock (MB) basis. Experiments show that our algorithm achieves about 5-30% bit-rate saving in the QP range of 28-36 without perceptual (visual) quality degradation.

Error concealment technique can improve subjective video quality in decoder when the video bitstream is corrupted during transmission. In this paper, a new error concealment algorithm for I-frames is proposed. To achieve perceptually comfortable results, image inpainting algorithm is adopted with structure information generated from edge information. In addition, mixed spatial-temporal inpainting scheme is also proposed for I-frames when the previous frame is available. Moreover, a block-based scheduling is also developed for the proposed algorithm to make it simpler for hardware implementation. Experimental results show that the proposed algorithm outperforms the official H.264 decoder and can deal with various error patterns. It also shows that it is suitable for hardware implementation in terms of on-chip memory requirement and predictable processing cycles.

Error Concealment techniques are widely used in video decoder with error-prone communication channels. In this paper, an enhanced edge-sensitive processing order for temporal error concealment algorithm is proposed. Side information of neighboring macroblocks of the corrupted macroblocks are considered to derive a suitable processing order for error concealment, and a new motion vector searching algorithm is also proposed for temporal error concealment. Experimental results prove that the processing order plays an important role in error concealment. With considering the processing order, the proposed algorithm outperforms existing algorithms in terms of PSNR and perceptual artifacts, and the improvement of 2.45dB in PSNR can be achieved compared with the same system with raster-scan order.

Error detection plays an important role in an error- robust video decoder. In this paper, a spatial-temporal error detection scheme for a video decoder is proposed. By considering inherently spatial and temporal similarities in video sequences, the visually corrected macroblocks in the decoded frames are detected by employing a set of error detection procedures, where one cross-boundary similarity index and one cross-frame similarity index are defined for spatial and temporal error detection, respectively. An adaptive threshold scheme is also proposed to make the proposed error detection method suitable for different video sequences. After being integrated with an H.264 decoder with error concealment techniques, the video quality improvement of 0.5-2.4 dB in PSNR is achieved. This method can also be integrated with other video codecs to improve the decoded video quality over noisy channels.