Sparsity is becoming arguably the most critical dimension to explore for efficiency and scalability as deep learning models grow significantly larger. Particularly, pruning is a common method to reduce redundant computations in attention-based and convolution-based models. The induced sparse matrix multiplication (SpMM) normally requires domain-specific hardware architecture (DSA) to eliminate unnecessary zero-valued computations. However, generating an optimal kernel code for SpMM on general-purpose and ISA-based spatial accelerators without changing the hardware architecture is still an open problem.In this paper, we propose a compiler plug-in named SpMMPlu, which can extend the representation and optimization ability for SpMM in current deep learning compiler frameworks that only support dense matrix multiplication. The key of SpMMPlu is a flexible intermediate representation— Sparse IR, representing the SpMM with various sparsity patterns based on meta-ops with a multi-level structure. Meta-op takes abstraction of the hardware intrinsic as its minimum granularity, and the powerful optimizers of existing NN compiler backends (e.g., Auto-schedule in TVM, AKG in MindSpore) can be easily reused for its computational scheduling and code generation. Moreover, we propose a two-step (segmentation & grouping) method to achieve an efficient Sparse IR for each sparsity pattern. Only three passes are added in SpMMPlu to provide an automatic solution for SpMM kernel code generation. We embed SpMMPlu into MindSpore and do experiments on NVIDIA V100 GPU and Huawei Ascend 910 to verify its effectiveness and scalability. The results show that with SpMMPlu, MindSpore can support various sparsity patterns and deliver a 1.93× (on V100 GPU) and 2.21× (on AScend 910) speedup averagely compared to the dense counterpart.

Binary Neural Networks (BNNs) have attracted tremendous attention in ReRAM-based Process-In-Memory (PIM) systems, since they significantly simplify the hardware-expensive peripheral circuits and memory footprint. Meanwhile, BNNs are proven to have superior bit error tolerance, which inspires us to make use of this capability in PIM systems whose memory bit-cell suffers from severe device defects. Nevertheless, prior works of BNN do not simultaneously meet the criterion that 1) achieving similar accuracy w.r.t its full-precision counterpart; 2) fully binarized without full-precision operation; and 3) rapid BNN construction, which hampers its real-world deployment. This work proposes the first framework called HAWIS, whose generated BNN can satisfy all the above criteria. The proposed framework utilizes the super-net pre-training technique and reinforcement-learning based width search for BNN generation. Our experimental results show that the BNN generated by HAWIS achieves 69.3% top-1 accuracy on ImageNet with ResNet-18. In terms of robustness, our method maximally increases the inference accuracy by 66.9% and 20% compared to 8-bit and baseline 1-bit counterparts under ReRAM non-ideal effects. Our-code is available at: https://github.com/DamonAtSjtu/HAWIS.

Abstract Speckle reduction is still an intractable task in ultrasound imaging field. Ultrasound speckle is usually described as multiplicative noise with its statistics following a Rayleigh or Gaussian distribution. To employ these two distributions effectively, the authors attempt to describe ultrasound speckle using a Weibull distribution, because it can include the Rayleigh distribution as a special case and also approximate a Gaussian distribution by varying its shape and scale parameters. The authors’ contribution in this paper is to propose a Weibull‐distribution‐based hybrid total variation (WHTV) method to reduce ultrasound speckle. The WHTV energy functional is convex and consists of a new data fidelity term and a new regularization term. The former is derived from the multiplicative Weibull model of ultrasound speckle based on the maximum likelihood criterion. The latter is a new edge‐weighted combination of the first‐ and second‐order total variation, with the advantage of preserving edges while alleviating the staircase effects. The minimization of the WHTV energy functional is implemented by the split Bregman algorithm. Experimental results on synthetic and real ultrasound images have demonstrated not only that the Weibull distribution is a better fitting model for the statistics of ultrasound speckle than other distributions such as Rayleigh, Gaussian, Gamma, and Nakagami, but also that the proposed WHTV method can achieve better despeckling performance than several state‐of‐the‐art variational methods.