This paper describes a joint response to the Call for Proposals by Samsung, Huawei, GoPro, and HiSilicon on Video Compression with Capability beyond HEVC/H.265, jointly issued by ITU-T SG16 Q.6 (VCEG) and ISO/IEC JTC1/SC29/WG11 (MPEG). In the proposed codec, the coding framework supports hierarchical splitting with binary and ternary trees and flexible coding order representations. Additionally, novel compression tools on inter/intra prediction, in-loop filtering, and entropy coding have been proposed. The proposed compression scheme provides significantly higher compression capability than the state-of-the-art HEVC/H.265 standard for SDR (Standard Dynamic Range) category while maintaining complexity acceptable for emerging applications. When all the proposed algorithmic tools are used, the proposed video codec achieves approximately 40% bit-saving for the SDR cetegory on average compared to HEVC/H.265 anchor.

AVS2 video standard is the next-generation video coding standard under the development of Audio Video coding Standard (AVS) workgroup of China. In this paper, the design of Sample Adaptive Offset (SAO) in AVS2 is presented. Considering the implementation issues, a shifted structure in which the SAO parameter region is shifted from the Largest Coding Unit (LCU) to the upper-left is adopted to make the SAO parameter region consistent with the processing region in implementation. Moreover, the category dependent offset is introduced in the edge type based on the statistical results to improve the offset coding and non-consecutive offset bands are adopted in the band type to optimize offset bands. The test results show that SAO achieves on average 0.3% to 1.4% luma coding gain in AVS2 common test conditions.

A concept of a quadtree based nonsquare block coding structure is presented in this paper for the emerging High Efficiency Video Coding standard, which includes a quadtree based asymmetric motion partitioning scheme and a nonsquare quadtree transform (NSQT) algorithm. Nonsquare motion partitioning in inter frame coding provides the possibility of getting more accurate prediction results by splitting one coding block into two nonsquare prediction blocks. Contrary to the traditional symmetric motion partitions (SMP), asymmetric motion partitions (AMP) are proposed to improve the coding efficiency, especially for the coding blocks with irregular object boundaries. NSQT is designed for nonsquare prediction blocks (SMP and AMP), which combines square and nonsquare transform blocks in a unified transform structure. It exploits the directional characteristic of an image block to improve the transform efficiency. The combination of nonsquare partitions and NSQT provides high coding flexibility and low implementation cost for both encoder and decoder design. Simulation results show that about 0.9%-2.8% bit-rate saving can be achieved in terms of different configurations, and subjective quality can also be improved.

This paper proposes a novel video compression scheme based on a highly flexible hierarchy of unit representation which includes three block concepts: coding unit (CU), prediction unit (PU), and transform unit (TU). This separation of the block structure into three different concepts allows each to be optimized according to its role; the CU is a macroblock-like unit which supports region splitting in a manner similar to a conventional quadtree, the PU supports nonsquare motion partition shapes for motion compensation, while the TU allows the transform size to be defined independently from the PU. Several other coding tools are extended to arbitrary unit size to maintain consistency with the proposed design, e.g., transform size is extended up to 64 × 64 and intraprediction is designed to support an arbitrary number of angles for variable block sizes. Other novel techniques such as a new noncascading interpolation Alter design allowing arbitrary motion accuracy and a leaky prediction technique using both open-loop and closed-loop predictors are also introduced. The video codec described in this paper was a candidate in the competitive phase of the high-efficiency video coding (HEVC) standardization work. Compared to H.264/AVC, it demonstrated bit rate reductions of around 40% based on objective measures and around 60% based on subjective testing with 1080 p sequences. It has been partially adopted into the first standardization model of the collaborative phase of the HEVC effort.

This paper proposes a video fingerprinting method based on a novel binary fingerprint obtained using a feature selection algorithm called the symmetric pairwise boosting (SPB). The binary fingerprints are obtained by filtering and quantizing perceptually significant features extracted from an input video clip. The SPB algorithm, which is a generalization of the conventional asymmetric pairwise boosting (APB), selects appropriate filters and quantizers from a class of candidate filters and quantizers in such a way that perceptually similar and dissimilar pairs of video clips are correctly classified as matching and non-matching pairs, respectively. The binary form of the novel fingerprint makes it conducive to an efficient database search, and the experimental results show that the proposed method outperforms the APB-based video fingerprinting methods in terms of both robustness and discriminability.

A novel binary audio fingerprint obtained by filtering and then quantizing the spectral centroids is proposed. A feature selection algorithm, coined pairwise boosting (PB), is used to determine the filters and quantizers by casting the fingerprinting problem of identifying a query audio clip into a binary classification problem. The PB algorithm selects the filters and quantizers which lead to accurate classification of matching and nonmatching audio pairs: a matching pair is an audio pair that should be classified as being identical, and a nonmatching pair is a pair that should be classified as being different. By iteratively reducing the classification error of both matching and nonmatching pairs, the PB algorithm improves both the robustness and discriminating ability. In our experiments, the proposed fingerprint outperformed previously reported binary fingerprints in terms of robustness and discriminating ability. In the experiment, we compared the performances of a number of distance measures.

This paper proposes a robust video fingerprinting method based on affine covariant regions. In video fingerprinting, a video clip is identified using short feature vectors referred to as fingerprints. In the proposed method, local fingerprints based on the centroid of gradient orientations are extracted from affine covariant regions detected in each key-frame. For the region detection, the maximally stable extremal region (MSER) detector which is considered to have high repeatability and low complexity is used. For reliable matching of the local fingerprints, only spatio-temporally consistent matches are taken into account. The experimental results show that the proposed method is robust against both geometric and non-geometric transformations.

This paper proposes a robust video fingerprinting method based on 2-Dimensional Oriented Principal Component Analysis (2D-OPCA) of affine covariant regions. The goal of video fingerprinting is to identify a video clip using perceptual features called fingerprints. In the proposed method, to achieve the robustness against geometric transformations, fingerprints are extracted from local regions co-variant with a class of affine transformations. The detected affine covariant regions are normalized geometrically and photometrically, and local fingerprints are extracted by applying a novel discriminant analysis algorithm, 2D-OPCA to the normalized regions. For the reliable matching of local fingerprints, only spatio-temporally consistent matches are taken into account. The experimental results show that the proposed method is robust against both geometric and non-geometric transformations.

Video fingerprints are feature vectors that uniquely characterize one video clip from another. The goal of video fingerprinting is to identify a given video query in a database (DB) by measuring the distance between the query fingerprint and the fingerprints in the DB. The performance of a video fingerprinting system, which is usually measured in terms of pairwise independence and robustness, is directly related to the fingerprint that the system uses. In this paper, a novel video fingerprinting method based on the centroid of gradient orientations is proposed. The centroid of gradient orientations is chosen due to its pairwise independence and robustness against common video processing steps that include lossy compression, resizing, frame rate change, etc. A threshold used to reliably determine a fingerprint match is theoretically derived by modeling the proposed fingerprint as a stationary ergodic process, and the validity of the model is experimentally verified. The performance of the proposed fingerprint is experimentally evaluated and compared with that of other widely-used features. The experimental results show that the proposed fingerprint outperforms the considered features in the context of video fingerprinting.

This paper proposes a high capacity reversible image watermarking scheme based on integer-to-integer wavelet transforms. The proposed scheme divides an input image into nonoverlapping blocks and embeds a watermark into the high-frequency wavelet coefficients of each block. The conditions to avoid both underflow and overflow in the spatial domain are derived for an arbitrary wavelet and block size. The payload to be embedded includes not only messages but also side information used to reconstruct the exact original image. To minimize the mean-squared distortion between the original and the watermarked images given a payload, the watermark is adaptively embedded into the image. The experimental results show that the proposed scheme achieves higher embedding capacity while maintaining distortion at a lower level than the existing reversible watermarking schemes.

Fingerprints are feature vectors that can uniquely characterize the video signal. The goal of a video fingerprinting system is to judge whether two videos have the same contents by measuring distance between fingerprints extracted from the videos. In this paper, a novel video fingerprinting method based on the centroids of gradient orientations is proposed. The centroid of gradient orientations is chosen due to its reliability and robustness against common video processing steps. A threshold used to reliably determine a fingerprint match is theoretically derived, and its validity is experimentally verified. The experimental results show that the proposed fingerprint is not only pairwise independent but also robust against common video processing steps.

The performance of a fingerprinting system, which is often measured in terms of reliability and robustness, is directly related to the features that the system uses. In this letter, we present a new audio-fingerprinting method based on the normalized spectral subband moments. A threshold used to reliably determine a fingerprint match is obtained by modeling the features as a stationary process. The robustness of the normalized moments was evaluated experimentally and compared with that of the spectral flatness measure. Among the considered subband features, the first-order normalized moment showed the best performance for fingerprinting.

For multimedia fingerprinting, it is crucial to extract relevant features that allow direct access to the distinguishing characteristics of a multimedia object. Features used for fingerprinting directly relate to the performance of the entire fingerprinting system. The paper proposes a novel audio fingerprinting method based on normalized spectral subband centroids. The spectral subband centroid is selected due to its resilience against equalization, compression, and noise addition. Both reliability and robustness issues in the fingerprinting system are addressed. Experimental results show that the proposed method is not only reliable, but also robust against various audio processing steps, including MP3 compression, equalization, random start, time-scale modification, and linear speed change.

A novel transcoding algorithm for the adaptive multi rate (AMR) codec and the enhanced variable rate codec (EVRC) is proposed. In contrast to the conventional tandem transcoding algorithm, the proposed algorithm transcodes the parameters of one codec to the other without synthesizing the speech. The proposed algorithm decodes the parameters of source codec from the input bitstream, and based on frame classification and mode decision, it appropriately transforms the parameters of source codec to those of the target codec in the parametric domain. Finally, the transformed parameters are encoded into a bitstream that is decodable by the target codec. The parameters transcoded by the proposed algorithm are line-spectral pair (LSP), pitch delay, fixed codevector, codebook gains, and frame energy. Evaluation results show that while reducing both the computational complexity and delay by 50%, the proposed algorithm produces speech quality equivalent to that of produced by the tandem transcoding algorithm. The general idea is not restricted to the AMR and EVRC but is applicable to various other code-excited linear prediction (CELP) based codecs.

In this paper, we propose an efficient rate selection algorithm that can be used to transcode speech encoded by any code excited linear prediction (CELP)-type codec into a format compatible with selectable mode vocoder (SMV) via direct parameter transformation. The proposed algorithm performs rate selection using the CELP parameters. Simulation results show that while maintaining similar overall bit-rate compared to the rate selection algorithm of SMV, the proposed algorithm requires less computational load than that of SMV and does not degrade the quality of the transcoded speech.

In this paper, a novel transcoding algorithm for the Selectable Mode Vocoder (SMV) and the G.723.1 speech coder is proposed. In contrast to the conventional tandem transcoding algorithm, the proposed algorithm converts the parameters of one coder to the other without going through the decoding and encoding process. The proposed algorithm is composed of four parts: the parameter decoding, Line Spectral Pair (LSP) conversion, pitch period conversion and rate selection. The evaluation results show that the proposed algorithm achieves equivalent speech quality to that of tandem transcoding with reduced computational complexity and delay.