We propose an end-to-end learned scalable multilayer feature compression method. Our proposed method is illustrated in Figure 1 , where f 1 ,… ,f n stand for deep features at different layers. The deep feature f n denoting base layer is first transformed and quantized into the latent ${\hat y_n}$ . The latent ${\hat y_n}$ is then inversely transformed to reconstruct the feature as f ˆ n . In addition, the latent ${\hat y_n}$ is also fed into the entropy model of the previous-layer feature f n−1 as conditional information for the enhancement layer. The entropy model of the feature f n−1 takes both ${\hat y_{n - 1}}$ and ${\hat y_n}$ as inputs to improve compression efficiency.

In this work, we investigate the joint optimization strategy in the scenario of video coding for machines (VCM). We formulated two kinds of joint optimization strategies, Opt_JA and Opt_JH , and compared them with the separate optimization strategy Opt_S. The three optimization strategies are illustrated in Fig. 1 . In Opt_S , we separately train the feature compression network with mean squared error (MSE). In Opt_JA , we optimize all modules jointly toward the person re-identification task. In Opt_JH , only the aggregation module and feature compression module are jointly optimized. The feature compression consists of two fully-connected (FC) layers and two batch normalization (BN) layers. Specifically, we set five compression ratios (CR): 256, 128, 64, 32, and 16.