This paper explores gaze prediction in dynamic 360° immersive videos, i.e., based on the history scan path and VR contents, we predict where a viewer will look at an upcoming time. To tackle this problem, we first present the large-scale eye-tracking in dynamic VR scene dataset. Our dataset contains 208 360° videos captured in dynamic scenes, and each video is viewed by at least 31 subjects. Our analysis shows that gaze prediction depends on its history scan path and image contents. In terms of the image contents, those salient objects easily attract viewers' attention. On the one hand, the saliency is related to both appearance and motion of the objects. Considering that the saliency measured at different scales is different, we propose to compute saliency maps at different spatial scales: the sub-image patch centered at current gaze point, the sub-image corresponding to the Field of View (FoV), and the panorama image. Then we feed both the saliency maps and the corresponding images into a Convolutional Neural Network (CNN) for feature extraction. Meanwhile, we also use a Long-Short-Term-Memory (LSTM) to encode the history scan path. Then we combine the CNN features and LSTM features for gaze displacement prediction between gaze point at a current time and gaze point at an upcoming time. Extensive experiments validate the effectiveness of our method for gaze prediction in dynamic VR scenes.

The high efficiency video coding (HEVC) standard has highly improved the coding efficiency by adopting hierarchical structures of coding unit (CU), prediction unit (PU), and transform unit (TU). However, enormous computational complexity is introduced due to the recursive rate-distortion optimization (RDO) process on all CUs, PUs and TUs. In this paper, we propose a fast and efficient mode decision algorithm based on the Neyman-Pearson rule, which consists of early SKIP mode decision and fast CU size decision. First, the early mode decision is modeled as a binary classification problem of SKIP/non-SKIP or split/unsplit. The Neyman-Pearson-based rule is employed to balance the rate-distortion (RD) performance loss and the complexity reduction by minimizing the missed detection with a constrained incorrect decision rate. A nonparametric density estimation scheme is also developed to calculate the likelihood function of the statistical parameters. Furthermore, an online training scheme is employed to periodically update the probability density distributions for different quantization parameters (QPs) and CU depth levels. The experimental results show that the proposed overall algorithm can save 65% and 58% computational complexity on average with a 1.29% and 1.08% Bjontegaard Delta bitrate (BDBR) increase for various test sequences under random access and low delay P conditions, respectively. The proposed overall scheme also has the advantage that it can make the trade-off between the RD performance and time saving by setting different values for the incorrect decision rate.

In High Efficiency Video Coding (HEVC), SKIP mode is an efficient inter prediction tool with high coding performance but low complexity. In this paper, an early SKIP mode decision algorithm is proposed to accelerate the encoding process. The rate-distortion cost (RD-cost) of SKIP mode is used as the decision criterion. A nonparametric density estimation scheme is employed to partition the SKIP RD-cost distribution space into high distinction region (HDR) and low distinction region (LDR). For a given Coding Unit (CU), if the SKIP RD-cost falls into the HDR, the SKIP mode is directly selected as the optimal mode. If the RD-cost maps into LDR, a Bayes risk minimization rule is adopted to ensure RD performance. The statistical parameters are updated according to different QPs and CU depths. Experimental results show that the proposed algorithm can reduce 47% of encoding time with only 0.34% of BD-bitrate increase on average.

The new video coding standard, High Efficiency Video Coding (HEVC), has been established to succeed the widely used H.264/AVC standard. However, an enormous amount of legacy content is encoded with H.264/AVC. This makes high performance AVC to HEVC transcoding in great need. This paper presents a fast transcoding algorithm based on residual and motion information extracted from H.264 decoder. By exploiting these side information, regions' homogeneity characteristic are analysed. An efficient coding unit (CU) and prediction unit (PU) mode decision strategy is proposed combing regions' prediction homogeneity and current encoding information. The experimental results show that the proposed transcoding scheme can save up to 55% of encoding time with negligible loss of coding efficiency, when compared to that of the full decoding and full encoding transcoder.

The H.264/AVC coding standard has achieved significant coding gain by using flexible block sizes inter prediction. However, the complexity of inter-mode decision is extremely high. In this paper, a fast inter-mode decision algorithm is presented by exploiting residual homogeneity for both macroblock and sub-macroblock levels. In this algorithm, the residual homogeneity indicators calculated in three directions are used to identify the block's homogeneous characteristic. Normalized motion vector activity is also considered to improve the performance. The experimental results illustrate that the proposed algorithm achieves about 49% of computational complexity saving in terms of encoding time, while with negligible quality degradation, compared to H.264/AVC reference software JM.

Motion estimation is the most computationally intensive element in a block based video codec. In this paper, a novel Predictive Intensive Direction Searching (PIDS) algorithm is proposed to reduce the computational load of H.264/AVC video codec. Based on the direction of predicted motion vector, the search area is adaptively divided into one region of intensive-direction-searching and several regions of' coarse-direction-searching. Then the extended hexagon search is used to refine the final optimal motion vector. The experimental results indicate that the proposed algorithm achieves a reduction of 15% motion estimation time, while incurring only 0.5% increment on the bite rate compared with that of UMHexagonS algorithm, which is adopted by the H.264/AVC reference software.