This paper describes a video coding technology proposal submitted by MediaTek in April 2018 in response to a joint call for proposals (CfP) issued by ITU-T VCEG and ISO/IEC MPEG. The proposal uses the conventional block-based hybrid coding approach with a breakthrough design of block structure, substantial improvements of coding tools, and new inventions of loop filtering techniques. First, the block structure is designed using a highly flexible partitioning scheme, where each coding tree unit in a picture is split into multiple coding units (CUs) by a recursive quaternary tree followed by a recursive binary-ternary tree. The prediction unit (PU) and transform unit (TU) concepts are unified with the CU concept, i.e., PU and TU are basically the same as CU. Moreover, inter prediction, intra prediction, transform, quantization, in-loop filtering, and entropy coding are all improved. Finally, new loop filtering techniques are invented, where the proposed convolutional neural network loop filtering is of particular interest. The proposed video codec achieves 43.81% average bit rate savings based on objective measures compared to the High Efficiency Video Coding (HEVC) anchors and is among the best-performing CfP responses both objectively and subjectively. It has been partially adopted into the working draft of the Versatile Video Coding (VVC) standard.

Movie summarization aims at condensing a full-length movie to a significantly shortened version that still preserves the movie's major semantic content. In this paper, we propose a learning-based movie summarization framework via role-community social network analysis and feature fusion. In our framework, scene-based movie summarization is formulated as a 0-1 knapsack problem, where the scene attention value for each significant scene is calculated as its “value” and the length of this scene is used as its “cost.” To identify the significance of each scene, we propose a learning-based approach to fuse the information derived from visual saliency (based on low-level features and high-level cognitive process for an input movie), high-level semantic analysis (based on the global and local social networks constructed from the movie), and user preferences. Our evaluation results show that in most test cases, the proposed method subjectively outperforms attention-based and role-based summarization methods and our previous role-community-based method in terms of semantic content preservation.

This paper presents an adaptive equalizer that converts the attenuated signal into Duobinary signaling scheme, combined with automatic Duobinary tracking technique to produce high quality Duobinary signal for simplifying the Duobinary decoding process and achieving higher data rate. The adaptive equalizer uses dual gain-mode topology that allows higher gain when the received signal is highly attenuated, and allows lower power consumption when the received signals pass through low-attenuation channel. A background offset cancellation loop circuit is added to increase the clock phase margin of the equalizer. The chip is fabricated in 0.18μm CMOS technology and operates at data rate of 10 Gb/s. The measurement results show that the equalizer recovers data properly for FR-4 trace with length ranging from 3 inches to 66 inches. The equalizer achieves clock phase margin of 58 % for 66-inch channel with BER less than 10-12and consumes power of 28.4 mW under high gain mode with 1.8V power supply.

Saliency detection is widely used to extract regions of interest in images for various image processing applications. Recently, many saliency detection models have been proposed for video in uncompressed (pixel) domain. However, video over Internet is always stored in compressed domains, such as MPEG2, H.264, and MPEG4 Visual. In this paper, we propose a novel video saliency detection model based on feature contrast in compressed domain. Four types of features including luminance, color, texture, and motion are extracted from the discrete cosine transform coefficients and motion vectors in video bitstream. The static saliency map of unpredicted frames (I frames) is calculated on the basis of luminance, color, and texture features, while the motion saliency map of predicted frames (P and B frames) is computed by motion feature. A new fusion method is designed to combine the static saliency and motion saliency maps to get the final saliency map for each video frame. Due to the directly derived features in compressed domain, the proposed model can predict the salient regions efficiently for video frames. Experimental results on a public database show superior performance of the proposed video saliency detection model in compressed domain.

Video summarization techniques aim at condensing a full-length video to a significantly shortened version that still preserves the major semantic content of the original video. Movie summarization, being a special class of video summarization, is particularly challenging since a large variety of movie scenarios and film styles complicate the problem. In this paper, we propose a two-stage scene-based movie summarization method based on mining the relationship between role-communities since the role-communities in earlier scenes are usually used to develop the role relationship in later scenes. In the analysis stage, we construct a social network to characterize the interactions between role-communities. As a result, the social power of each role-community is evaluated by the community's centrality value and the role communities are clustered into relevant groups based on the centrality values. In the summarization stage, a set of feasible summary combinations of scenes is identified and an information-rich summary is selected from these candidates based on social power preservation. Our evaluation results show that in at most test cases the proposed method achieves better subjective performance than attention-based and role-based summarization methods in terms of semantic content preservation for a movie summary.

This paper presents an adaptive cable equalizer employing a time-domain approach called the phase detection technique in 0.13μm CMOS technology. The equalizer can compensate a 24-inch microstrip on an FR4 PCB, which has an 18dB channel loss at 5GHz. Meanwhile it also avoids offset-sensitive rectifiers which normally exist in conventional equalizers. The proposed adaptive cable equalizer consumes 39mW excluding the output buffer from a 1.5-V supply voltage and the measured BER is less than 10-13.

This work proposes an adaptive equalizer employing a new overshoot control for achieving optimized equalization. The proposed overshoot detection scheme obviates the need for slicers to achieve high-speed operation with low power dissipation. The circuit adapts to FR4 trace lengths for up to 40 inches. The measurement results show that the equalizer achieves a good eye performance while consuming 40mW from a 1.8V supply.

Saliency detection is widely used to extract the regions of interest in images. Many saliency detection models have been proposed for videos in the uncompressed domain. However, videos are always stored in the compressed domain such as MPEG2, H.264, MPEG4 Visual, etc. In this study, we propose a video saliency detection model based on feature contrast in the compressed domain. Four features of luminance, color, texture and motion are extracted from DCT coefficients and motion vectors in the video bitstream. The static saliency map of video frames is calculated based on the luminance, color and texture features, while the motion saliency map for video frames is computed by motion feature. The final saliency map for video frames is obtained through combining the static saliency map and motion saliency map. Experimental results show good performance of the proposed video saliency detection model in the compressed domain.

In this paper, we propose a novel image retargeting algorithm based on the sensitivity-tuned visual significance map which is composed of a saliency map and a gradient map. We develop a new saliency detection model based on the human visual sensitivity and amplitude spectrum of image patches. We use a coherent normalization based fusion method to combine the saliency map and the gradient map to generate the visual significance map. The seam carving technique is adopted for image retargeting, based on the sensitivity-tuned visual significance map. Experiment results show that the proposed algorithm outperforms the relevant state-of-the-arts image retargeting algorithms significantly.

Video retargeting from a full-resolution video to a lower resolution display will inevitably cause information loss. Content-aware video retargeting techniques have been studied to avoid critical visual information loss while resizing a video. Maintaining the spatio-temporal coherence of a retargeted video is very critical on visual quality. Camera motions and object motions, however, usually make it difficult to maintain temporal coherence using existing schemes. In this paper, we propose the use of a panoramic mosaic to guide the scaling of corresponding regions of video frames in a video shot to ensure good temporal coherence. In the proposed method, after aligning video frames in a shot to a panoramic mosaic constructed for the shot, a global scaling map for these frames is derived from the panoramic mosaic. Subsequently, the local scaling maps of individual frames are derived from the global map and is further refined according to spatial coherence constraints. Our experimental results show that the proposed method can effectively maintain temporal coherence so as to achieve good visual quality even a video contains camera motions and object motions.

We conduct subjective tests to evaluate the performance of scalable video coding with different spatial-domain bit-allocation methods, visual attention models, and motion feature extractors in the literature. For spatial-domain bit allocation, we use the selective enhancement and quality layer assignment methods. For characterizing visual attention, we use the motion attention model and perceptual quality significant map. For motion features, we adopt motion vectors from hierarchical B-picture coding and optical flow. Experimental results show that a more accurate visual attention model leads to better perceptual quality. In cooperation with a visual attention model, the selective enhancement method, compared to the quality layer assignment, achieves better subjective quality when an ROI has enough bit allocation and its texture is not complex. The quality layer assignment method is suitable for region-wise quality enhancement due to its frame-based allocation nature.

We propose a fast mode decision algorithm to reduce the computational complexity of adaptive GOP structure (AGS) in the scalable extension of H.264/AVC. AGS can improve the coding efficiency of the scalable extension of H.264. It, however, needs to perform motion-compensated temporal filtering (MCTF) of all possible GOP sizes, leading to much higher computation than the fixed GOP structure. In our proposed algorithm, after performing the MCTF with the maximum GOP size, we utilize two features to decide whether to perform the remaining MCTFs of sub-GOPs and mode selection. Experimental results show that the proposed algorithm can significantly reduce unnecessary MCTF computation for AGS, while maintaining good coding efficiency.