Layer pruning is a viable technique for compressing large language models while achieving acceleration proportional to the pruning ratio. In this work, we identify that removing any layer induces a magnitude gap in hidden states, and demonstrate that a simple compensation operation leads to superior performance in iterative layer pruning. This key observation motivates us to propose Prune&Comp, a novel, plug-and-play iterative layer pruning scheme that leverages magnitude compensation to mitigate such gaps in a training-free manner. Specifically, we first estimate the magnitude gap of layer removal and then eliminate it by rescaling the remaining weights offline. We further demonstrate the advantages of Prune&Comp in improving the stability of iterative pruning. When integrated with an iterative prune-and-compensate loop, Prune&Comp consistently enhances existing layer pruning metrics. For instance, when 5 layers of LLaMA-3-8B are pruned with the prevalent Taylor+ metric, Prune&Comp reduces PPL from 512.78 to 16.34 and retains 90.57% of the original performance across 9 question-answering tasks, outperforming the baseline by 24.72%.

Quantization has enabled the widespread implementation of deep learning algorithms on resource-constrained Internet of Things (IoT) devices, which compresses neural networks by reducing the bit-width of their parameters. However, most quantization methods invade privacy as they require real training datasets for calibration or fine-tuning. As a solution, zero-shot quantization (ZSQ) has emerged as a paradigm to quantize neural networks without accessing training datasets. Most employ data generation schemes to synthesize calibration data for knowledge transfer from the full-precision networks to the quantized ones. For privacy-protected and resource-constrained IoT devices, achieving optimal deployment necessitates the strategic integration of synthetic data generation and low-bit-width quantization techniques. However, when it comes to the lower bit-width case in ZSQ, we observe that the discrepancy between the full-precision network and the quantized network tends to widen significantly, hindering the knowledge transfer, which is attributed to the three following challenges: 1) hard logits matching with wide discrepancy; 2) unstable feature alignment with huge quantization error; and 3) synthetic data with low diversity. To address these issues, this article presents S-ZSQ, a novel ZSQ framework with two-pronged strategies that enhances both knowledge transfer and synthetic data generation, which enables low-bit-width quantized network to derive more soft feature-infused hints from the full-precision network. We achieve significant improvements on classification tasks, including CIFAR-10/100 and ImageNet-1k, with fewer fine-tuning epochs, particularly in scenarios involving low-bit-width quantization. For example, in the 3-bit ResNet-18/ResNet-50 case, we outperform AdaDFQ by 8.08%/11.16% in top-1 accuracy on ImageNet-1k.

To deploy neural networks on clinical edge devices, quantization is the most commonly used method to compress the models, which requires a calibration set of hundreds of real images. However, due to privacy concerns, the scarcity of private histopathological images hinders the application of quantization. To address this issue, we develop HIQ, a novel one-shot quantization framework for histopathological image classification networks, which requires only one real image per class for calibration. To compensate for data scarcity, sample BNS alignment is introduced to generate synthetic images with similar distribution to the real ones. To improve the diversity of synthetic images, fine-grained diversity enhancement that provides fine-grained enhancement intensity for different classes and network layers is proposed, based on the observation of the class-wise and layer-wise fine-grained data. Finally, the asymptotic enhancement strategy is highlighted to achieve a trade-off between inter-class distance and intra-class diversity of synthetic images, based on the insight of the smaller inter-class distance of histopathological images than that of natural ones. Extensive experiments on the BRACS dataset show that our method achieves an extremely low accuracy loss even compared to the full precision model in low-bit cases and maintains robustness when missing classes of real images.

Histopathology analysis is the gold standard for medical diagnosis. Accurate classification of whole slide images (WSIs) and region-of-interests (ROIs) localization can assist pathologists in diagnosis. The gigapixel resolution of WSI and the absence of fine-grained annotations make direct classification and analysis challenging. In weakly supervised learning, multiple instance learning (MIL) presents a promising approach for WSI classification. The prevailing strategy is to use attention mechanisms to measure instance importance for classification. However, attention mechanisms fail to capture inter-instance information, and self-attention causes quadratic computational complexity. To address these challenges, we propose AMD-MIL, an agent aggregator with a mask denoise mechanism. The agent token acts as an intermediate variable between the query and key for computing instance importance. Mask and denoising matrices, mapped from agents-aggregated value, dynamically mask low-contribution representations and eliminate noise. AMD-MIL achieves better attention allocation by adjusting feature representations, capturing micro-metastases in cancer, and improving interpretability. Extensive experiments on CAMELYON-16, CAMELYON-17, TCGA-KIDNEY, and TCGA-LUNG show AMD-MIL's superiority over state-of-the-art methods.

Quantization is an effective way to compress neural networks. By reducing the bit width of the parameters, the processing efficiency of neural network models at edge devices can be notably improved. Most conventional quantization methods utilize real datasets to optimize quantization parameters and fine-tune. Due to the inevitable privacy and security issues of real samples, the existing real-data-driven methods are no longer applicable. Thus, a natural method is to introduce synthetic samples for zero-shot quantization (ZSQ). However, the conventional synthetic samples fail to retain the detailed texture feature distributions, which severely limits the knowledge transfer and performance of the quantized model. In this paper, a novel ZSQ method, TexQ is proposed to address this issue. We first synthesize a calibration image and extract its calibration center for each class with a texture feature energy distribution calibration method. Then, the calibration centers are used to guide the generator to synthesize samples. Finally, we introduce the mixup knowledge distillation module to diversify synthetic samples for fine-tuning. Extensive experiments on CIFAR10/100 and ImageNet show that TexQ is observed to perform state-of-the-art in ultra-low bit width quantization. For example, when ResNet-18 is quantized to 3-bit, TexQ achieves a 12.18% top-1 accuracy increase on ImageNet compared to state-of-the-art methods. Code at https://github.com/dangsingrue/TexQ.