Conventional pruning techniques typically focus on evaluating a single structure in the network, such as the convolutional layer or batch normalization layer, to identify pruning targets. However, this approach fails to effectively leverage the potential of all structures within each layer of the network. In order to comprehensively consider the various structures in each layer, we propose a novel method called Soft Hybrid Filter Pruning using a Dual Ranking Approach (DR-SHFP), which builds upon Soft Filter Pruning (SFP) by introducing a dual-ranking approach. DR-SHFP incorporates a ranking system that assigns a rank to each filter in a collaborative manner, taking into account both convolutional layers and batch normalization layers. By simultaneously evaluating both types of layers, our method captures more information from the layer structures, overcoming the limitations of single-structure evaluation. Consequently, DR-SHFP can identify and select filters more effectively for pruning, leading to improved performance. Experimental results demonstrate the effectiveness of DR-SHFP on benchmark datasets such as CIFAR-10, CIFAR-100, and Tiny-ImageNet. The proposed method outperforms other soft pruning methods, showcasing its capability to achieve excellent performance in various settings.

This study proposes a hybrid filter pruning method based on linear region analysis. Our approach combines the advantages of cluster pruning and norm-based filter pruning by introducing thresholds based on Euclidean distance and norm distance. We also incorporate the linear region analysis approach in neural network architecture search to estimate the performance of trained model architectures. This enables us to efficiently search for the optimal pruned structure with corresponding thresholds for effective model compression.

The paper proposes a new approach called Multi-Objective Filter Pruning (MOFP), which formulates the filter pruning of deep learning models as a multi-objective optimization problem. The proposed approach applies the Non-Dominated Sorting Genetic Algorithm II to solve the problem and the Asymmetric Gaussian Distribution (AGD) for population initialization. Compared with existing methods, MOFP shows competitiveness in terms of a balance of objectives between compression rates, computing power, and prediction accuracy. In addition, the search result of MOFP is a Pareto Front, which eliminates the need for multiple searches to obtain architectures with different compression rates, significantly improving overall search efficiency. The results show that the use of AGD for population initialization can enhance the search process by effectively exploring the search space, leading to higher quality results.