The ability to efficiently switch from one pre-encoded video stream to another (e.g., for bitrate adaptation or view switching) is important for many interactive streaming applications. Recently, stream-switching mechanisms based on distributed source coding (DSC) have been proposed. In order to reduce the overall transmission rate, these approaches provide a merge mechanism, where information is sent to the decoder, such that the exact same frame can be reconstructed given that any one of a known set of side information (SI) frames is available at the decoder (e.g., each SI frame may correspond to a different stream from which we are switching). However, the use of bit-plane coding and channel coding in many DSC approaches leads to complex coding and decoding. In this paper, we propose an alternative approach for merging multiple SI frames, using a piecewise constant (PWC) function as the merge operator. In our approach, for each block to be reconstructed, a series of parameters of these PWC merge functions are transmitted in order to guarantee identical reconstruction given the known SI blocks. We consider two different scenarios. In the first case, a target frame is first given, and then merge parameters are chosen, so that this frame can be reconstructed exactly at the decoder. In contrast, in the second scenario, the reconstructed frame and the merge parameters are jointly optimized to meet a rate-distortion criteria. Experiments show that for both scenarios, our proposed merge techniques can outperform both a recent approach based on DSC and the SP-frame approach in H.264, in terms of compression efficiency and decoder complexity.

In the current video coding standards, rate-distortion optimization (RDO) plays an important role in achieving best tradeoff between the perceived distortion and transmission rate. It is widely used in all kinds of encoder decisions, including block mode decision, motion vector selection and so on. Generally, the sum of absolute difference (SAD) or the sum of square difference (SSD) is used as the distortion measurement. However, it is well known that both of them cannot always reflect the perceptual quality of the encoded video. In this paper, an objective quality measurement structural similarity (SSIM) index is proposed as the distortion measurement in the RDO framework for video coding standards. By fully exploiting the relationship between SSIM and mean square error (MSE), the SSIM-based RDO framework can be approximated by the original SSD-based RDO framework with only a scaling of the Lagrange multiplier. Experimental results show that the proposed method outperforms the latest H.264 codec and also the state-of-the-art SSIM-based RDO video codec.

Non-camera captured images (also known as compound image) contain a mixture of camera-captured natural images and computer-generated graphics and texts. Nowadays, there are more and more applications calling for non-camera captured image/video compression scheme. However, current video coding standards, which are designed for natural video, treat non-camera captured video less carefully. For example, the state-of-the-art video coding standard High Efficiency Video Coding (HEVC) may blur or even remove edges in text/graphic region. A lot of schemes are proposed to preserve direction property of texts and graphics, such as palette-based intra coding. In this paper, a novel palette coding scheme is proposed for palette-based intra coding in HEVC. The palette in a block is predicted from an adaptive palette template, which records the statistical non-local spatial correlation of an image. Every block chooses its own palette using the palette template as the prediction in a rate-distortion optimized manner. Experimental results show that the proposed scheme can achieve up to 5.2% bit-rate saving compared to the state-of-the-art palette-based coding scheme in HEVC.

Rate-distortion-optimized (RDO) intermode decision is one of the most effective tools that greatly improves the coding performance of modern coding standards, for example, H.264/AVC and HEVC. However, RDO intermode decision also leads to extremely intense computation. To reduce the complexity, a fast intermode decision algorithm is presented in this paper. Mathematical analysis of inter prediction residual is performed which explicitly shows the impact of edge information and motion characteristics to the prediction accuracy. Moreover, It is shown that with fixed quantization step that minimizing of R-D costs over different partition types is equivalent to minimizing the variance of transformed residual which can be expressed by motion vector and edge gradient components. In consequence, the complex calculation of rate and distortion is replaced by simple pre-analysis of video content. The repetition of motion estimation (ME) and entropy coding process are avoided. Inspired by the theoretical analysis, a fast inter mode decision algorithm is proposed. Experimental results show that the fast method achieves considerable complexity reduction with negligible coding performance degradation.

Motion estimation (ME) is one of the key elements in video coding standard which eliminates the temporal redundancy by using a motion vector (MV) to indicate the best match between the current frame and reference frame. A coarse to fine process is taken to find the best MV. First of all, integer-pixel ME finds a coarse MV and followed by the sub-pixel ME around the best integer-pixel point. The sub-pixel ME plays an important role in improving the coding efficiency. However, the computational complexity of searching one sub-pixel point is much higher than the integer-pixel point searching because of the interpolation and Hadamard transform operation. In this paper, an accurate optimal sub-pixel position prediction algorithm is presented. With the information of the 8 neighboring integer-pixel points, the optimal sub-pixel position is predicted directly without explicitly solving model parameters. Moreover, an outlier rejection scheme is applied to improve the robustness of the proposed algorithm. Experimental results show that the proposed algorithm outperforms the state of the art interpolation-freesub-pixel ME algorithms.

The ability to efficiently switch from one pre-encoded video stream to another is a valuable attribute for a variety of interactive streaming applications, such as switching among streams of the same video encoded in different bit-rates for real-time bandwidth adaptation, or view-switching among videos capturing the same dynamic 3D scene but from different viewpoints. It is well known that intra-coded I-frames can be used at switch boundaries to facilitate stream-switching. However, the size of an I-frame is large, making frequent insertion impractical. A recent proposal towards a more efficient stream-switching mechanism is distributed source coding (D-SC), which exploits worst-case correlation between a set of potential predictor frames in the decoder buffer (called side information (SI) frames) and a target frame to lower encoding rate. However, the conventional use of bit-plane and channel coding means the encoding and decoding complexity of DSC frames is large. In this paper, we pursue a novel approach to the stream-switching problem based on the concept of “signal merging”, using piecewise constant (p-wc) function as the merge operator. Specifically, we propose a new merge mode for a code block, where for each k-th transform coefficient in the block, we encode appropriate step size and horizontal shift parameters at the encoder, so that the resulting floor function at the decoder can map corresponding coefficients from any SI frame to the same reconstructed value, resulting in an identically merged signal. The selection of shift parameter per coefficient, as well as coding modes between intra and merge per block, are optimized in a rate-distortion (RD) optimal manner. Experiments show encouraging coding gain over a previous implementation of DSC frame at low-to mid-bitrates at reduced computation complexity.

Images encoded at low bit rate usually exhibit visually annoying coding artifacts, which are commonly referred as blocking artifacts. In this paper, a novel weighted adaptive total variation method is proposed to remove the blocking artifacts. Based on the analysis of image coding process and block-based discrete cosine transform (BDCT) image properties, image deblocking is formulated as an optimization problem which is solved through approximating the objective function by a set of convex functions. Experimental results show that the proposed method can achieve better objective and subjective quality performance compared to other deblocking algorithms.

The advance in multimedia technologies have resulted in an explosive growth of pictures in personal computers and in cloud. Typically many pictures taken in the same occasion are similar. The cost to store and transmit them can be very significant. Thus it is important to find an efficient method to store these pictures. This paper proposed a compression scheme for similar images. Our approach is to arrange all the similar images into tree structure then apply video coding technique along each branch. To maximize the inter-image correlation between adjacent photos, we consider the minimum spanning tree (MST) subjecting to a maximum depth limit to ensure fast access to all images. This structure is encoded by the latest video coding technique High Efficiency Video Coding (HEVC), which is reported to has advantage in high definition video/image compression. It also supports deleting, adding and modifying images. Experiments show that the proposed method saved 75% space comparing to JPEG format.

Merge/skip mode is one of the most important inter prediction tools adopted in the High Efficiency Video Coding (HEVC) standard which is the state-of-the-art video coding standard. It is very efficient in reducing the side information for the blocks within the same object. However, it is difficult for parallel encoding and decoding due to the data dependency problem between neighboring prediction units (PU). Furthermore, different shapes and positions of PUs would result in different definition of the merge/skip candidate list (MCL), which would lead to potentially extra hardware cost and is not easy to be efficiently implemented by the hardware. To deal with this problem, two reconfigurable hardware-friendly MCL construction schemes are proposed in this paper. The first scheme which is called unified MCL (UMCL) uses one candidate list for all PUs inside the motion estimation region (MER), which is regarded as the basic parallel processing unit for the hardware realization. The second scheme which is named boundary MCL (BMCL) allows different candidate lists for the PUs on the boundary of MER. Both of the two schemes can have flexible parallel degree based on the requirement specification. Experimental results show that UMCL reduces the hardware complexity significantly with little coding performance degradation and BMCL achieves significant coding gain while maintaining the hardware complexity.

The High Efficiency Video Coding (HEVC) is the next generation video coding standard beyond H.264/AVC. Compared with only up to 9 modes for intra prediction in H.264/AVC, HEVC provides 35 intra prediction modes (IPM) to improve coding efficiency, which inevitably poses a huge complexity burden to the encoder. To speed up the HEVC encoder, a novel fast mode decision (FMD) algorithm for HEVC intra prediction is proposed. In the proposed algorithm, we analyzed the costs generated by rough mode decision (RMD), which has already been incorporated in the HM software. We found that the RMD costs listed by mode number generally follow the same trend with the rate-distortion optimization (RDO) costs. Further, the local salient modes, whose RMD costs have a significant drop compared with adjacent modes, tend to be promising competitors for the optimal mode. Based on these observations, we further reduced the number of the candidates for the RDO process. Experimental results show that our proposed algorithm achieves averagely 19.0% (up to 33.6%) encoding time saving whilst causing negligible RD performance loss (0.4% BD-Rate increase on average) compared with HM 7.0 anchor.

Scalable video coding (SVC), which is an extension of H.264/AVC video coding standard, was introduced to provide scalability in different dimensions for adaptation to heterogeneous network and terminals. After the finalization of the new video coding standard called High Efficiency Video Coding (HEVC), the effort of the standardization committee has been redirected to the investigation of the scalable extension of HEVC. In addition to the basic inter-layer texture prediction mechanism, several other coding tools were proposed for coding performance improvement. Among those coding tools, the one called generalized residual prediction (GRP) scheme achieves most significant coding gain while the consumption of computational power is also huge. In this paper, the GRP mechanism is formulated and analyzed in detail. In addition, the combination of GRP mechanism with merge mode in HEVC is proposed for simplification of the existing GRP mechanism. Results show a better trade-off could be achieved by greatly reducing computational complexity while maintaining most of the coding gain.

In this paper, we address the frame-level dependent bit allocation (DBA) problem in hybrid video coding. In most existing methods, the DBA solution is achieved at the expense of high, sometimes even unbearable, computational complexity because of the multipass coding involved. Motivated by this, we propose a model-based approach as an attempt to solve this problem analytically. Leveraging the predictive nature in hybrid video coding, we develop a novel interframe dependency model (IFDM), which enables a quantitative measure of the coding dependency between the current frame and its reference frame. Based on the IFDM, the buffer-constrained frame-level DBA problem is carefully formulated. Finally, the model-based DBA method called IFDM-DBA is derived, in which successive convex approximation techniques are employed to convert the original optimization problem into a series of convex optimization problem s of which the optimal solutions can be obtained efficiently. Experimental results suggest that the proposed IFDM-DBA method can achieve up to a 23% bitrate reduction over the JM reference software of H.264.

Motion estimation (ME) is one of the most time consuming parts in video coding standard. As fast integer-pixel ME algorithm becoming more and more powerful, it is important to develop fast sub-pixel ME algorithm since the computational complexity of sub-pixel ME compared to integer-pixel ME has become relatively significant. In this paper, a novel fast sub-pixel ME algorithm is proposed. This algorithm first approximates the error surface of the sub-pixel position by a second order function and predicts the minimum point by minimizing the function at half-pixel accuracy. Then another second order approximation within a smaller area which is determined by the previous step is modeled to predict the best sub-pixel position. Experimental results show that the proposed method can reduce the sub-pixel search points significantly with negligible quality degradation.

Motion estimation (ME) is one of the key elements in video coding standard which eliminates the temporal redundancies between successive frames. In recent international video coding standards, sub-pixel ME is proposed for its excellent coding performance. Compared with integer-pixel ME, sub-pixel ME needs interpolation to get the value in sub-pixel position. Also, Hadamard transform will be applied in order to achieve better performance. Therefore, it is becoming more and more critical to develop fast sub-pixel ME algorithms. In this paper, a novel fast sub-pixel ME algorithm is proposed which makes full use of 8 neighboring integer-pixel points. This algorithm models the error surface in sub-pixel position by a second order function with five parameters two times to predict the best sub-pixel position. Experimental results show that the proposed method can reduce the complexity significantly with negligible quality degradation.

Medical imaging applications produce a huge amount of similar images. Instead of compressing each image individually, set redundancy compression (SRC) methods remove the inter image redundancy and reduce storage. However, in the previous SRC methods — -MMD, MMP and Centroid methods, the prediction templates for extracting set redundancy are not very efficient, especially when image sets are very large with several clusters. In this paper, inspired by face recognition techniques, a novel lossless SRC method is derived based onDCT pyramid multi-level low frequency template. The approximation subband is used as a prediction template for each image to calculate the residue. Intra prediction is also used to reduce the entropy of the residues. Experiments with 3 sets of MR brain images demonstrate the efficiency of our proposed algorithm in respect to bits/pixel (bpp).

Adaptive quantization matrix design for different block sizes is one of the possible methods to improve the RD performance in video coding and has recently attracted the focus of many researchers. In this paper, we first analyze the shortcomings of the evenly distributed distortion method which was proposed recently. In order to tackle these problems, we propose two modified methods, method I with relaxed distortion constraints and method II is iterative boundary distortion minimization problem considering variance adaptively. Both problems can be solved using convex optimization effectively and efficiently. Simulations have been conducted based on HM4.0, which is the reference software of the latest High Efficiency Video Coding (HEVC). Simulation results show the effect of our proposed methods. Both methods show their significance when evaluated by RD performance.

In video coding standard, motion estimation (ME) always plays an important role in reducing temporal redundancies at the expense of higher computational complexity. Many fast ME algorithms have been proposed to reduce the coding complexity. Some papers focus on applying specific search patterns to reduce the search points within a fixed search range (SR). But there are only a few of them trying to reduce the size of SR. In this paper, an adaptive SR algorithm is presented. Cauchy distribution is used to model the SR for one frame and the information of motion vector differences in the neighboring blocks is used to adjust the SR for a particular block. Experimental results show that the proposed algorithm can reduce the size of SR significantly with negligible quality degradation.

In this paper, we propose a simple but effective adaptive bilateral filter considering local characteristics. The presented method exploits the local gaussian gradient information of the processing image and applies bilateral filter with changing the range filter parameter $\sigma_{r}$ adaptively. The proposed adaptive bilateral filter could preserve more details of the image compared with original bilateral filter in edge or texture regions. What is more, it is more robust to the noise. Also compared with the original bilateral filter and the state-of-art wavelet de-noising method, the proposed method also can effectively remove signal noise while preserving the original structure.