Abstract This paper designs a novel method to reduce the coding complexity of 3D‐HEVC encoder by utilizing the properties of human visual perception. Two vision‐oriented edge detections are proposed: for colour texture detection, the authors adopt the Just‐Noticeable Distortion (JND); for depth map, the authors combine the Sample Adaptive Offset (SAO) and the Just Noticeable Depth Difference (JNDD) model. The authors also analyse the properties of colour texture and depth map to classify the coding tree unit (CTU) into various kinds of types, including complex‐edge CTU, moderate‐edge CTU and homogeneous CTU. Besides, fast mode decisions and early termination criteria are performed individually on each type of CTUs according to their characteristics. Especially for those CTUs with more edge information, the proposed projection‐based fast mode decision and residual‐based early termination preserve important colour texture while speeding up the coding at the same time. The proposed vision‐oriented algorithm reduces 31.981% of the overall average coding time with only 1.580% BD‐Bitrate increase. Experimental results show that the proposed algorithm can provide considerable time‐saving while still maintain the video quality, which outperforms the previous researches.

3D-HEVC (The 3D Extension of High Efficiency Video Coding) is the newest 3D video coding standard, which enriches multimedia applications with the video format of multi-view plus depth. For the depth map coding in 3D-HEVC, the advanced coding tools enhance the coding efficiency of the depth map and the quality of the synthesized view. However, the time consumption and complexity of 3D-HEVC also increase significantly. This paper utilizes the characteristics of human visual system to propose a fast algorithm based on visual perception for the acceleration of the depth intra coding of 3D-HEVC. The depth map is segmented into different regions by Otsu's auto-thresholding. The dominate edge direction is categorized for each prediction unit. We detect the perceptual edge based on just noticeable depth difference model to extract the area that may affect the visual perception. According to depth map segmentation and edge distribution, we reduce the corresponding intra angular modes and determine whether to perform depth modelling mode. We also incorporate the boundary continuity and rate-distortion cost thresholding to propose the fast coding unit decision. The experimental results show that the proposed algorithm eliminates 53.09% of the depth coding time with only 0.15% BD-BR on average. The coding performance of the proposed algorithm outperforms the previous works significantly.

High-efficiency video coding encoders provide great improvements in coding efficiency and can also support higher resolution and multiple coding tools. The new coding structures such as coding unit (CU) and prediction unit (PU) have helped a lot, but the computational complexity is much higher than those of previous standards. This paper proposes a fast algorithm combining with CU and PU early termination decisions to reduce computational demand. Based on the analytic results, we can set up an adaptive threshold that can be obtained for early termination. Meanwhile, we also develop an adaptive search range determination according to the motion vector (MV). Compared with HM 12.0, our proposed method achieves an approximate 57% time saving, whereas the average Bjøntegaard-Delta Bit-rate (BDBR) increase is only 0.43%. In addition, our fast algorithm outperforms the previous works in both coding speed and coding performance.

In this paper, a whole frame loss error concealment algorithm for three-dimensional video coding is proposed. The main concept of the proposed algorithm is to extrapolate the motion vectors for concealing a current error block by jointly considering the available motion vector and the depth information. In addition, the depth information is adopted to help the derivation of reference pixels for concealing errors in the case that suitable motion vectors cannot be obtained by the motion vector extrapolation process alone. Simulation results demonstrate that our proposed algorithm can achieve up to 0.52 dB PSNR, as well as subjective quality improvement, compared to previous work.

Multiview video coding (MVC) has attracted great attention from industries and research institutes. MVC is used to encode stereoscopic video streams for 3D playout systems such as 3D television, digital cinema, and IP network applications. MVC is an extended version of H.264/AVC that improves the performance of multiview videos. Yet, when compared with single-view video coding, MVC consumes much more time when encoding large amounts of data. Speed-up algorithms, therefore, are essential for realizing related applications. This paper presents a fast mode decision algorithm to avoid the high computational complexity of MVC. The proposed approach aims to reduce candidate modes and make mode decision process more efficient. The minimum and maximum values of rate-distortion cost (RD cost) in the previously encoded view are used to compute a threshold for each mode in the current view. Compared with joint multiview video coding, the experimental results demonstrate that the proposed algorithm provides an average of 79% in time savings with negligible bit rate increase and peak signal-to-noise ratio decrease.

This paper presents a region-of-interest (ROI) segmentation technique to facilitate the video transcoding. In the low bit-rate transmission, ROI video coding can maintain better visual quality based on the properties of the human visual system (HVS). In this paper, to aim this purpose, the Bayesian theorem is utilized to automatically determine the moving foreground objects by using motion intensity analysis and skin color information for inter coded frames. Experimental results show that the proposed method can extract ROI macroblocks (MBs) effectively.

A prediction error prioritizing-based normalized partial distortion search algorithm for fast motion estimation is proposed in this letter. The distortion behavior of each pixel in a macroblock is first analyzed to point out the priority/order of sum of absolute difference calculation. Afterward, the normalized partial distortion search algorithm is applied for half-stop of the distortion calculation. In addition, a dynamic search range decision algorithm is adopted for automatically changing the size of the search range to further increase the motion estimation speed. The computational complexity can be reduced significantly through the proposed algorithm, though leaving a PSNR degradation that could be dismissed.

Thenewestvideo coding standard called scalable video coding (SVC) provides broad applications in multimedia communications. SVC encoder consumes great computational complexity when compared to previous video coding standards. This paper presents a fast mode decision algorithm that speeds up the SVC encoding process through probabilistic analysis. The mode of the enhancement layer is first predicted by statistical analysis. Afterward, Bayesian theorem is utilized to detect whether the prediction mode of the current macroblock is the best or not. The mode is further predicted and refined by the Markov process. Experimental results show that the proposed algorithm significantly reduces computational complexity with negligible peak signal-to-noise ratio degradation and bitrate increase in the enhancement layers.

In this paper, an adaptive video de-interlacing algorithm based on motion characteristics detection jointly considering the advantages of three existing schemes is proposed. In the proposed method, the difference of the previous field pixels and the current field pixels are calculated. The fast or slow of the motion of the current frame is determined by the difference of the previous and the current field pixels. Afterwards, the appropriate scheme such as the weave, line average or edge line average de-interlacing schemes is selected and performed on the selected field. Through the proposed algorithms, the performance of the proposed method in terms of subjective and objective qualities can be significantly improved.

Region-of-interest (ROI) determination is very important for video processing and it is desirable to find a simple method to identify the ROI. Along this direction, this paper investigates a user attention model based on visual rhythm analysis for automatic determination of ROI in a video. The visual rhythm, which is an abstraction of a video, is a thumbnail version of a video by a 2-D image that captures the temporal information of a video sequence. Four sampling lines, including diagonal, anti-diagonal, vertical, and horizontal lines, are employed to obtain four visual rhythm maps in order to analyze the location of the ROI from video data. Via the variation on visual rhythms, object and camera motions can be efficiently distinguished. As for hardware design consideration, the proposed scheme can accurately extract ROI with very low computational complexity for real-time applications. The promising results from the experiments demonstrate that the moving object is effectively and efficiently extracted. Finally, we present a way to use flexible macroblock ordering in combination with ROI determination as a preprocessing step for H.264/AVC video coding, and experimental results show the quality of ROI regions is significantly enhanced.

This paper investigates a user attention model based on the visual rhythm analysis for automatically determining the region-of-interest (ROI) in a video. The visual rhythm, an abstraction of a video, is a thumbnail version of a fully video by a 2D image that captures the temporal information of a video sequence. Four sampling lines, including diagonal, anti-diagonal, vertical and horizontal lines, are employed to obtain four visual rhythm maps in order to analyze the location of the ROI from video data. Via the variation on visual rhythms, object and camera motions can be efficiently distinguished. The proposed scheme can extract the ROI accurately with very low computational complexity. The promising results from the experiments demonstrate that the moving object is effectively and efficiently extracted.

H.264 is a new video coding standard which contains some novel coding functions and provides more coding efficiency than previous standards. However, video streams are sensitive to channel errors that often degrade the visual quality of the decoded sequence, not only corrupt the current frame but also propagate to subsequent frames. In order to solve this problem, the error concealment in the decoder is very important and useful, neither bit rate nor delay will increase during the transmission. In this paper, we propose an error concealment method by utilizing the temporal correlation of the lost macroblock and its neighboring macroblocks to reconstruct the damaged data. Simulation results show that both of the subjective and objective performances of the proposed techniques are superior to conventional temporal concealment methods for H.264 standard

The MPEG-4/AVC/H.264 video coding standard adopts various coding schemes such as multiple reference frames and variable block sizes for motion estimation. Hence, MPEG-4/AVC/H.264 provides gains in compression efficiency of up to 50% over a wide range of bit rates and video resolutions compared to previous standards. However, these features result in a considerable increase in encoder complexity, mainly regarding to mode decision and motion estimation. The proposed algorithms use the stored motion vectors to compose the motion vector without performing the full search in each reference frame. Therefore, the proposed algorithms can obtain an average speed up ratio of four for encoding, thus benefiting from the prediction of the motion vector for the reference frames in advance and maintaining good performance. Any fast search algorithm can be utilized to further largely reduce the computational load.

In this paper, a fast motion estimation algorithm for variable block-size by using a motion vector merging procedure is proposed for H.264. The motion vectors of adjacent small blocks are merged to predict the motion vectors of larger blocks for reducing the computation. Experimental results show that our proposed method has lower computational complexity than full search, fast full search and fast motion estimation of the H.264 reference software JM93 with slight quality decrease and little bit-rate increase

In this paper, new methods for error concealment of shape information are proposed for compressed MPEG-4 bit-streams transmitted over error prone channels. For spatial domain, the proposed approach employs the Bezier curve fitting to approximate and outline the intra-video object plane (I-VOP). For temporal domain, the proposed scheme uses the weighted side matching criterion to reconstruct the inter-video object planes (P-/B-VOPs). The proposed algorithms get better subjective and objective qualities than previous works.

Compressed video bitstream is sensitive to channel errors that may degrade the reconstructed pictures severely. Error concealment is an effective approach to reduce the impact of errors, and it is implemented at the decoders. Its aim is to mask the effect of the missing blocks to create image or video of subjectively acceptable quality by using the spatial or/and temporal correlation. This paper proposes a temporal error concealment method that exploits the temporal redundancy of a video sequence using a block-matching approach. A recursive block-matching algorithm, based on the two-step block-matching principle, is presented to refine the concealed video. The proposed algorithm can also conceal the multiple erroneous slices. The proposed algorithm outperforms the algorithms presented in previous studies.

In multimedia applications, it is often necessary to adapt the bit rates of the coded video bit streams to the available bandwidths of various communication channels. Because different networks may have different bandwidths, a gateway can include a transcoder to adapt the video bit rates in order to provide video services to users on different networks. In transcoding, motion estimation is usually not performed in the transcoder because of its heavy computational complexity. To speed up the operation, a video transcoder usually reuses the decoded motion vectors (MVs) from the incoming bitstream. Previously, the bilinear interpolation and forward dominant vector selection methods were proposed to compose the missing MVs for reuse. We propose a new algorithm, called the activity dominant vector selection, which utilizes the quantized discrete cosine transform coefficients of residual blocks for composing a MV from the incoming ones. In addition, a motion-vector refinement algorithm called variable step-size search is presented. The performance can be improved while maintaining low computational complexity for a reduced frame-rate video transcoder.

A new error resilience algorithm for recovering the lost blocks or motion vectors is presented. It combines the overlapped motion compensation and the side match criterion to make the effect of lost blocks subjectively imperceptible. The side match criterion takes advantage of the spatial contiguity and interpixel correlation of image to select the best-fit replacement among the motion vectors of spatially contiguous candidate blocks. Particularly, to mask the blocking artifacts, we incorporate an overlapping technique to create a subjectively closer approximation to the true error-free image.

Most video sequence coding systems use block motion compensation to remove temporal redundancy for video compression due to the regularity and simplicity. A new error concealment algorithm for recovering the lost or erroneously received motion vectors is presented. It combines the overlapped motion compensation and the side match criterion to make the effect of lost motion vectors subjectively imperceptible. The side match criterion takes advantage of the spatial contiguity and interpixel correlation of image to select the best-fit replacement among the motion vectors of spatially contiguous candidate blocks. Particularly, to mask the blocking artifacts, we incorporate an overlapping technique to create a subjectively closer approximation to the true error-free image.

Among the various transform techniques for image compression, the discrete cosine transform (DCT) is the most popular and effective one in practical image and video coding applications, such as high-definition television (HDTV). We develop a novel 8/spl times/8 two-dimensional (2-D) discrete cosine transform/inverse discrete cosine transform (DCT/IDCT) architecture based on the direct 2-D approach and the rotation technique. The computational complexity is reduced by taking advantage of the special attribute of a complex number. Both the parallel and the folded architectures are proposed. Unlike other approaches, the proposed architecture is regular and economically allowable for VLSI implementation. Compared to the row-column method, less internal wordlength is needed in order to meet the error requirement of IDCT, and the throughput of the proposed architecture can achieve two times that of the row-column method with 30% hardware increased.

In this paper, a new video coding algorithm suitable for the very low bit rate video application is presented. It takes advantage of the prior knowledge of the image type to segment the image into different regions, then codes each region with different coding criterion and method according to the different importance. An adaptive region-classified vector quantization strategy is also exploited in this algorithm. With the segmentation of the frame and high correlation between frames, better codebooks of vector quantisation are constructed to improve the quality. According to the simulation results, acceptable quality at about 10 kbit/s can be obtained for the typical test sequences.

A novel and efficient block-matching motion estimation criterion called minimized maximum error (MiniMax) is considered. The proposed method can save hardware area about 15% with acceptable video performance. A chip which combines the MiniMax matching criterion and the one-dimensional full search algorithm is presented. The ASIC is motivated by the need of the intensive computational demand to perform motion estimation in real time. The proposed single chip can match the applications of H.261 and MPEG international standards. Chip cascading is allowed for larger searching range applications.>

A new hardware-oriented algorithm called the one-dimensional full search (1DFS) is presented for block-matching motion estimation in video compression. The simulation for this algorithm follows H.261 and MPEG international standards. In MPEG simulation, structures with 1-, 2- and 3-frame interpolation are compared. The performance of 1DFS is superior to that of other fast search algorithms. And it has more regular data flow, data reuse and less control overhead. It is an alternative for 2D full search block matching and achieves a good compromise between computational complexity and performance. With competent performance and reasonable computation complexity, the proposed method is more suitable for real-time hardware realization of a VLSI motion estimator for video applications.>