This paper presents a novel method to train the quantized feature with the action recognition task jointly. A quantization and inverse-quantization layers are introduced to the 3D CNN. The quantization and the action recognition loss functions are minimized jointly. That is, the method aims to learn the feature not only to improve action recognition accuracy but also reduce the information loss of the quantization. The framework is shown in Fig. (1).

Existing compression artifacts reduction methods aim to restore images on pixel-level, which can improve the human visual experience. However, in many applications, large-scale images are collected not for visual examination by humans. Instead, they are used for many high-level vision tasks usually by Deep Neural Networks (DNN). In this paper, we find that these methods have limited performance improvements to high-level tasks, even bring negative effects. Therefore, inspired by the teacher-student network framework, we propose a compression artifacts reduction framework (ARF) for DNN. In addition, we generalize our method to the unsupervised setting (U-ARF) where the corresponding original images are unavailable in training. Extensive experiments indicate the proposed methods can help DNNs improve performance on the highly compressed images significantly.

Short videos, ranging from a few seconds to a few minutes, have become popular in many video production. So we need efficient coding tools to compress these videos. AV1 is an emerging open-source and royalty-free video compression format jointly developed and finalized in early 2018 by the Alliance for Open Media. Many coding tools are introduced into AV1 to improve compression performance. However, some coding tools may lead to performance loss to encode these short videos. This paper presents a coding tools performance evaluation of AV1 on short videos. We design an Off experiment by turning off a single coding tool, and the anchor is AV1 with all coding tools on. We list the impact of each open and closed coding tool on the overall performance of the short video.

A common framework of the action recognition is to collect the videos from different cameras into a cloud center firstly, and then perform the 3D CNN on the cloud server. Although directly, this framework will bring a huge burden to the cloud server and video transmission. To handle this challenge, the "front-cloud" collaborative processing architecture can be used. The most import issue is to compress the feature from 3D CNN effectively without significant loss of accuracy. We propose logarithmic quantization with a maximum value threshold and HEVC inter encoding for 3D CNN features. Experimental results on ResNet-50 and InceptionV1 show that the features can be represented by only 1 bit without significant loss of accuracy. The compression ratio of the quantized 1 bit features using HEVC inter coding can reach to 5000 times and the loss of accuracy is less than 1%.

In the modern video coding standards, transform coefficient coding takes large portion of output bit-stream. The absolute value of coefficients are usually represented with high-efficiency variable-length code, while the signs are binarized with 1-bit fixed-length code, which ignores the apriority constraints on sign combinations beyond signs. In this paper, we present a fast and efficient sign inference algorithm for improving the coding efficiency of coefficients signs. By considering the correlation of pixels across the transform block boundary, we define a criterion of continuity to constrain the combination space of signs in the block, and shorten the bits will be used. A given number of signs are hidden during bypass coding, and inferred with the high-precision criterion. Then the infer-error is coded with arithmetic coding. Meanwhile a sign bit redefining strategy is introduced, which is based on the invariants of the linear orthogonal separable transform. The strategy ensures that the codec is able to choose up to 4 bits for inferring each block, and avoids the complex decision mechanism introduced by rate distortion optimization. Experiment results show that about 0.6% in average and up to 1.1% BD-rate saving is archived and only less than 1% encoding time complexity increased compared with HEVC baseline.