In this paper, we propose a novel motion vector processing approach based on the temporal analysis of motion vector reliability for motion-compensated frame interpolation. We first address the problems of conventional methods that find motion by minimizing the difference between the forward and backward predictions. The ghost artifacts can be reduced due to this procedure, but the resulting motion vectors may not represent the actual motion. Therefore, we propose analyzing the temporal variations of the bidirectional prediction difference along the motion trajectory to detect temporally unreliable motion vectors and further improve the accuracy of the bidirectional motion vector processing. Experimental results show that the proposed method can effectively detect unreliable motion vectors and outperforms other methods in terms of visual quality, and structure similarity.

In this paper, we investigate color distribution around object edges and further exploit this information for motion vector reliability analysis. Since the motion vectors are often estimated only based on luminance component, the resulting motion vectors may become unreliable for the areas with similar intensity. By making use of the color information for the received residual energy calculation, these unreliable motion vectors can effectively be identified. Moreover, the motion boundaries can be easily detected due to stronger residual energy distribution on the object boundaries. This characteristic can also be used to assist the motion vector processing in motion-compensated frame interpolation so that the interpolated object structures can be well maintained. Experimental results show that using same motion vector processing approach but with additional consideration on color information outperforms other methods.

In this paper, we address the problems of unreliable motion vectors that cause visual artifacts but cannot be detected by high residual energy or bidirectional prediction difference in motion-compensated frame interpolation. A correlation-based motion vector processing method is proposed to detect and correct those unreliable motion vectors by explicitly considering motion vector correlation in the motion vector reliability classification, motion vector correction, and frame interpolation stages. Since our method gradually corrects unreliable motion vectors based on their reliability, we can effectively discover the areas where no motion is reliable to be used, such as occlusions and deformed structures. We also propose an adaptive frame interpolation scheme for the occlusion areas based on the analysis of their surrounding motion distribution. As a result, the interpolated frames using the proposed scheme have clearer structure edges and ghost artifacts are also greatly reduced. Experimental results show that our interpolated results have better visual quality than other methods. In addition, the proposed scheme is robust even for those video sequences that contain multiple and fast motions.

In this paper, a novel correlation-based motion vector processing method is proposed for motion compensated frame interpolation. We first address the problem of unreliable motion vectors due to low correlation. Unlike other motion vector processing methods using vector median filter, we proposed using bidirectional prediction difference to select the best motion vectors with constraint on increasing motion vector correlation. Furthermore, to reduce the blockiness artifacts while maintaining edges, we use an adaptive vector averaging filter to obtain a finer motion field by taking motion vector correlation into account. Experimental results show that the proposed scheme outperforms other methods in terms of visual quality and PSNR performance.

In this paper, we address the potential issues in bidirectional motion compensated frame interpolation when the received motion vector field is directly used. Based on this motion vector analysis, we therefore propose using bidirectional motion vector processing method to eliminate the ghost artifacts around the moving objects. The artifacts caused by large motion vector magnitude can be effectively removed by correcting motion vectors bidirectionally. Moreover, since the proposed motion vector selection process chooses the best motion vector from adjacent motion vectors, the computational complexity can be greatly reduced comparing to motion estimation. We also demonstrate how to obtain clearer object edges by allowing displacement adjustment during the bidirectional motion vector processing. Experimental results show that the proposed algorithm outperforms other methods in terms of visual quality.

In this paper, a novel, low-complexity motion vector processing algorithm at the decoder is proposed for motion-compensated frame interpolation or frame rate up-conversion. We address the problems of having broken edges and deformed structures in an interpolated frame by hierarchically refining motion vectors on different block sizes. Our method explicitly considers the reliability of each received motion vector and has the capability of preserving the structure information. This is achieved by analyzing the distribution of residual energies and effectively merging blocks that have unreliable motion vectors. The motion vector reliability information is also used as a prior knowledge in motion vector refinement using a constrained vector median filter to avoid choosing identical unreliable one. We also propose using chrominance information in our method. Experimental results show that the proposed scheme has better visual quality and is also robust, even in video sequences with complex scenes and fast motion.

In this paper, a novel multi-stage motion vector processing algorithm at the decoder is proposed for motion compensated frame interpolation. We address the problems of discontinuous edges and deformed structures in an interpolated frame by explicitly considering reliability of each received motion vector. By hierarchically refining motion vectors with different block sizes, the proposed method is capable of preserving structure information. Experimental results show that the proposed scheme outperforms other methods in terms of visual quality and PSNR, and it is also robust when video sequences have complex scenes and fast motion.

This paper discusses the disadvantages of several existing objective quality evaluation methods for frame interpolation techniques. Samples show that these disadvantages lead the objective quality measurement inconsistent with what humans perceive. Based on these observations, a new metric designed to evaluate the performance of frame interpolation techniques is proposed. This metric combines the severity of interpolation artifacts and several human visual factors into a single quality score. Final implementation shows that the proposed metric out-performs other commonly used metrics.

In this paper, we address the problem of motion compensated frame interpolation (MCFI) at the decoder by analyzing received motion vectors (MVs) and residual error. In the proposed method, the skipped frames are generated at the decoder by using received information from the bitstream. We combine a hierarchical motion vector correction algorithm and a residual-energy constrained median filter to obtain true motion. Experimental results show that the proposed motion vector processing algorithm improves the visual quality of interpolated frames, especially in the motion transition area and the motion boundary.

In this paper, a novel motion compensated frame interpolation (MCFI) algorithm by merging blocks that have unreliable motion vectors (MVs) based on their residual errors is proposed. Unlike the conventional methods that find true motion using smaller blocks and vector median filter, we proposed to find one single motion vector to represent a group of adjacent macroblocks (MBs) where the conventional MCFI methods are likely to fail, likely to fail. The proposed method is able to preserve the structure of different objects and their edge information, without requiring complicated edge detection and object-based motion estimation. Experimental results show that the proposed scheme improves both visual quality and PSNR, especially in the areas with different motions and the motion boundary