Recent results have revealed an interesting observation in a trained convolutional neural network (CNN), namely, the rank of a feature map channel matrix remains surprisingly constant despite the input images. This has led to an effective rank-based channel pruning algorithm [23], yet the constant rank phenomenon remains mysterious and unexplained. This work aims at demystifying and interpreting such rank behavior from a frequency-domain perspective, which as a bonus suggests an extremely efficient Fast Fourier Transform (FFT)-based metric for measuring channel importance without explicitly computing its rank. We achieve remarkable CNN channel pruning based on this analytically sound and computationally efficient metric, and adopt it for repetitive pruning to demonstrate robustness via our scheme named Energy-Zoned Channels for Robust Output Pruning (EZCrop), which shows consistently better results than other state-of-the-art channel pruning methods. The codes and Appendix are publicly available at: https://github.com/ruilin0212/EZCrop.

We introduce a new kind of linear transform named Deformable Butterfly (DeBut) that generalizes the conventional butterfly matrices and can be adapted to various input-output dimensions. It inherits the fine-to-coarse-grained learnable hierarchy of traditional butterflies and when deployed to neural networks, the prominent structures and sparsity in a DeBut layer constitutes a new way for network compression. We apply DeBut as a drop-in replacement of standard fully connected and convolutional layers, and demonstrate its superiority in homogenizing a neural network and rendering it favorable properties such as light weight and low inference complexity, without compromising accuracy. The natural complexity-accuracy tradeoff arising from the myriad deformations of a DeBut layer also opens up new rooms for analytical and practical research. The codes and Appendix are publicly available at: https://github.com/ruilin0212/DeBut.

As deep learning neural networks (DNNs) advance and increase in computational complexity, particularly in terms of memory cost, it becomes difficult to implement DNNs in fixed-point memory-sparse environments (e.g. integrated circuits in consumer electronics). Thus, the training of DNNs must be reformulated to balance the hardware costs needed to represent the often millions of parameter weights in such machine learning models. This paper proposes a novel optimization approach that simultaneously minimizes complexity (total memory) and maximizes accuracy. Specifically, a bit-depth complexity penalty is induced to urge the DNN model towards a state of lower memory using a numerical gradient in optimization iterations. Experimental results on the MNIST (Mixed National Institute of Standards and Technology) handwritten digit classification benchmark demonstrate a minimal loss (1%) in DNN accuracy with a significant reduction (37%) of memory cost on average.

Film grain noise (FGN) is generated by the procedure of capturing pictures using photographic film. Images with FGN are subjectively pleasing. However, FGN is difficult to compress and its pleasant features are difficult to preserve when the images are resized. So in literature, FGN is extracted first, then regenerated for the processed noise-free images. In this paper a new method is proposed to generate FGN. In contrast to some other models which generate FGN in spatial domain, our method captures the statistic feature of FGN in frequency domain. FGN is signal dependent and the signal independent scaled noise image (SNI) is obtained by scaling FGN by corresponding noise-free image. We model each discrete cosine transform (DCT) coefficient of SNI as a Gaussian random variable. The Gaussian model parameters can be estimated from the stack of all the blocks in SNI based on stationary assumption. Experimental results show that proposed model recovers FGN with similar visual and spectrum properties to the original FGN. We also apply proposed model in superresolution and the quality of resultant images are improved.

Example-based super-resolution (SR) approaches mostly reconstruct and optimize the high-resolution (HR) image according to objective criteria such as imaging model. However, in consumer electronics applications the ultimate goal is better subjective visual effect rather than higher objective texture visibility. In this paper, we propose a SR method using perceptual visibility optimization (PVO). A human visual system (HVS) preference model is built based on just noticeable distortion (JND) threshold that considers the structural regularity of the texture. Following the scale-invariant self-similarity (SiSS) based SR reconstruction, an iterative backprojection (IBP) procedure coupled with the proposed model adaptively enhances the texture visibility to match the HVS's preference. Experimental results show the proposed approach has competitive quality and lower computational complexity compared with several state-of-the-art SR approaches.

Example-based super-resolution (SR) attracts great interest due to its wide range of applications. However, these algorithms usually involve patch search in a large database or the input image, which is computationally intensive. In this paper, we propose a scale-invariant self-similarity (SiSS) based super-resolution method. Instead of searching patches, we select the patch according to the SiSS measurement, so that the computational complexity is significantly reduced. Multi-shaped and multi-sized patches are used to collect sufficient patches for high-resolution (HR) image reconstruction and a hybrid weighting method is used to suppress the artifacts. Experimental results show that the proposed algorithm is 20~1,800 times faster than several state-of-the-art approaches and can achieve comparable quality.