Compressive autoencoders (CAEs) play an important role in deep learning-based image compression, but large-scale CAEs are computationally expensive. We propose a framework with three techniques to enable efficient CAE-based image coding: 1) Spatially-adaptive convolution and normalization operators enable block-wise nonlinear transform to spend FLOPs unevenly across the image to be compressed, according to a transform capacity map. 2) Just-unpenalized model capacity (JUMC) optimizes the transform capacity of each CAE block via rate-distortion-complexity optimization, finding the optimal capacity for the source image content. 3) A lightweight routing agent model predicts the transform capacity map for the CAEs by approximating JUMC targets. By activating the best-sized sub-CAE inside the slimmable supernet, our approach achieves up to 40% computational speed-up with minimal BD-Rate increase, validating its ability to save computational resources in a content-aware manner.

In this paper, we introduce an efficient hardware design for the third generation audio video coding standard (AVS3) based entropy coding, which satisfies the need of the emerging 8K video applications. The proposed hardware architecture is effectively established by partitioning functional modules for better dataflow organization. First, we customize the binarizer for massive parallelism. Second, by fully exploiting hardware sharing, a hardware/software co-design approach is initiated to optimize the pipelined throughput of the entropy coding engine. Third, we give an efficient buffer management method for sub-module coordination to ensure continuous bitstream generation. The proposed hardware design for entropy coding is the first work for the latest AVS3 video coding standard. The performance of the design is benchmarked based on the field-programmable gate array (FPGA) IP, which demonstrates the superior efficiency of hardware-accelerated entropy coding for streaming UHD media by achieving better design tradeoffs than the existing methods for previous standards.

The hardware-accelerated real-time compression of 8K Ultra-High-Definition (UHD) video is an exemplary application that empowered by the latest video coding standard. However, the coding tools added to the recently released third-generation audio video coding standard (AVS3) greatly increase the coding complexity, which seriously hinders the efficient implementation of hardware encoder. In order to break the known bottleneck, this paper presents the first open source software library for 8K UHD video coding hardware implementation, namely OpenHardwareVC. Specifically, based on the analysis of the original AVS3 software algorithm, we provide the hardware acceleration designs of the four major coding stages, including coding unit (CU) partition, intra prediction, transform and entropy coding, in this library. Simulation results on Xilinx VU440 FPGA show that the real-time compression of 8K UHD videos at 30 frames per second (fps) can be easily supported based on software-described modules packaged in this library. The release of this library is quite favorable for the hardware design and system implementation of UHD video coding, which is also beneficial to the promotion of the new coding standard. The open source library for OpenHardwareVC is available at https://git.openi.org.cn/OpenHardwareVC.

In recent years, automated channel pruning methods have been proposed to search optimal sub-networks from cumbersome neural network models. Among all the efforts, differentiable methods stand out as an efficient solution to the substructure search problem. However, the early differentiable channel pruning method relies on a predefined probability model, which implicitly poses an obstacle to the optimization. Moreover, the importance of architecture alignment across the bi-level optimization may be underestimated. In this paper, a novel search algorithm for the optimal pruned model is proposed by overcoming the aforementioned deficiencies. Follow-up experiments demonstrate the superiority of the proposed method in terms of efficiency and performance.

View synthesis (VS) for light field images is a very time-consuming task due to the great quantity of involved pixels and intensive computations, which may prevent it from the practical three-dimensional real-time systems. In this article, we propose an acceleration approach for deep learning-based light field view synthesis, which can significantly reduce calculations by using compact-resolution (CR) representation and super-resolution (SR) techniques, as well as light-weight neural networks. The proposed architecture has three cascaded neural networks, including a CR network to generate the compact representation for original input views, a VS network to synthesize new views from down-scaled compact views, and a SR network to reconstruct high-quality views with full resolution. All these networks are jointly trained with the integrated losses of CR, VS, and SR networks. Moreover, due to the redundancy of deep neural networks, we use the efficient light-weight strategy to prune filters for simplification and inference acceleration. Experimental results demonstrate that the proposed method can greatly reduce the processing time and become much more computationally efficient with competitive image quality.