This paper presents a coding unit (CU) depth-decision algorithm using neural networks to reduce the computational overhead of High Efficiency Video Coding (HEVC). The coding tree unit (CTU) of HEVC has a quad-tree structure, and its computational complexity is considerably high because it searches an optimal CU depth from the upper to lower depth recursively and exhaustively. In the proposed method, neural networks are used to predict the CTU depth. A database for neural networks is constructed, which considers both the image and encoding properties of the CU. It consists of the image data representing the image value of the CU, the vector data based on the encoding information of the CU, and the labels indicating whether the CU is divided. By using both properties of the CU, high test accuracy can be achieved. It is completely separated from the test sequence used for encoding and can be configured to use a sequence with various resolutions, motions, and contents for diverse CUs. We also design a neural-network architecture and perform training. The architecture consists of the convolution and pooling layers for analyzing the image property of the CU. The feature map is concatenated with the vector data and trained by fully connected layers in order to analyze the encoding property of the CU. Finally, a fast CU depth-decision algorithm is designed based on the trained neural networks. When the result of the neural network inference with the current CU depth is non-split, the operation on the lower CU depth is skipped. The experimental results show that the proposed method can reduce the computational overhead by 61.77% on average, and by a maximum of 73.45% with 3.91% Bjøntegaard-Difference-bitrate (BD rate) degradation.

This paper presents an optimized parallel algorithm for the next-generation video codec High Efficiency Video Coding (HEVC). The proposed method provides maximized parallel scalability by exploiting two levels of parallelism: 1) frame level and 2) task level. Frame-level parallelism is exploited using a graph that efficiently provides a parallel coding order of the frames with complex reference dependencies. The proposed reference dependency graph is generated at runtime by a novel construction algorithm that dynamically analyzes the configuration of the HEVC codec. Task-level parallelism is exploited to provide further scalability to frame-level parallelization. A pipelined execution is allowed for independent tasks, which are defined by dividing and categorizing a single coding process into multiple types of tasks. The proposed parallel encoder and decoder do not suffer from loss in coding efficiency because neither constraints nor modification in coding options are required. The proposed parallel methods result in an average encoding speedup of 1.75 and the aggressive method that exploits additional frame-level parallelism achieved 6.52 speedup using eight physical cores.

This paper presents a feature-based frame rate up-conversion algorithm which provides more comfortable visual experience by exploiting true motion of the objects. By considering the movement of the objects rather than the pixel values, the proposed method can create interpolated frames to reflect true movement of the video contents. We first find local features within a frame by using a feature detection algorithm. Then, the local features are matched between adjacent frames and are clustered to form an object region. The interpolated frame is created by using the perspective transformation, which enables to adequately track the dynamic movement of the defined objects. The proposed scheme efficiently resolves the blocking artifact problem and presents outstanding visual quality compared to the conventional block-based motion compensated interpolation algorithm.

The demand for real-time processing of robust feature detection is one of the major issues in the computer vision field. In order to comply with the requirements, in this paper a parallelization and optimization method to effectively accelerate SURF is proposed. The proposed parallelization method is developed based on a workload analysis of SURF in terms of various aspects, focusing in particular on the load balancing problem. First, the average parallel workload is divided into identical portions using the vertical partitioning method. Then, the load imbalance problem is further resolved using the dynamic partition balancing method. In addition, an optimization method is proposed together with the parallelization method to find and exclude redundant operations in SURF, thus effectively accelerating the feature detection operation when the proposed parallelization method is applied. The proposed method shows a maximum speedup of 19.21 compared to the single threaded performance on a 24-core system, achieving a maximum of 83.80 fps in a real-machine experiment, enabling real-time processing.