Deep neural networks for image super-resolution (SR) have demonstrated superior performance. However, the large memory and computation consumption hinders their deployment on resource-constrained devices. Binary neural networks (BNNs), which quantize the floating point weights and activations to 1-bit can significantly reduce the cost. Although BNNs for image classification have made great progress these days, existing BNNs for SR still suffer from a large performance gap between the FP SR networks. To this end, we observe the activation distribution in SR networks and find much larger pixel-to-pixel, channel-to-channel, layer-to-layer, and image-to-image variation in the acti-vation distribution than image classification networks. However, existing BNNs for SR fail to capture these variations that contain rich information for image reconstruction, leading to inferior performance. To address this problem, we propose SCALES, a binarization method for SR networks that consists of the layer-wise scaling factor, the spatial re-scaling method, and the channel-wise re-scaling method, capturing the layer-wise, pixel-wise, and channel-wise variations efficiently in an input-dependent manner. We evaluate our method across different network architectures and datasets. For CNN-based SR networks, our binarization method SCALES outperforms the prior art method by 0.2dB with fewer parameters and operations. With SCALES, we achieve the first accurate binary Transformer-based SR network, improving PSNR by more than 1dB compared to the baseline method.

State space models (SSMs) like Mamba have recently attracted much attention. Compared to Transformer-based large language models (LLMs), Mamba achieves linear computation complexity with the sequence length and demonstrates superior performance. However, Mamba is hard to accelerate due to the scattered activation outliers and the complex computation dependency, rendering existing LLM accelerators inefficient. In this paper, we propose LightMamba that co-designs the quantization algorithm and FPGA accelerator architecture for efficient Mamba inference. We first propose an FPGA-friendly post-training quantization algorithm that features rotation-assisted quantization and power-of-two SSM quantization to reduce the majority of computation to 4-bit. We further design an FPGA accelerator that partially unrolls the Mamba computation to balance the efficiency and hardware costs. Through computation reordering as well as fine-grained tiling and fusion, the hardware utilization and memory efficiency of the accelerator get drastically improved. We implement LightMamba on Xilinx Versal VCK190 FPGA and achieve 4.65~6.06 x higher energy efficiency over the GPU baseline. When evaluated on Alveo U280 FPGA, LightMamba reaches 93 tokens/s, which is 1.43 x that of the GPU baseline.

Stochastic computing (SC) has emerged as a promising computing paradigm for neural acceleration. However, how to accelerate the state-of-the-art Vision Transformer (ViT) with SC remains unclear. Unlike convolutional neural networks, ViTs introduce notable compatibility and efficiency challenges because of their nonlinear functions, e.g., softmax and Gaussian Error Linear Units (GELU). In this paper, for the first time, a ViT accelerator based on end-to-end SC, dubbed ASCEND, is proposed. ASCEND co-designs the SC circuits and ViT networks to enable accurate yet efficient acceleration. To overcome the compatibility challenges, ASCEND proposes a novel deterministic SC block for GELU and leverages an SC-friendly iterative approximate algorithm to design an accurate and efficient softmax circuit. To improve inference efficiency, ASCEND develops a two-stage training pipeline to produce accurate low-precision ViTs. With extensive experiments, we show the proposed GELU and softmax blocks achieve 56.3% and 22.6% error reduction compared to existing SC designs, respectively, and reduce the area-delay product (ADP) by 5.29 x and 12.6 x, respectively. Moreover, compared to the baseline low-precision ViTs, ASCEND also achieves significant accuracy improvements on CIFARI0 and CIFARI00.

Language models have shown promising performance on the task of translating natural language questions into SQL queries (Text-to-SQL). However, most of the state-of-the-art (SOTA) approaches rely on powerful yet closed-source large language models (LLMs), such as ChatGPT and GPT-4, which may have the limitations of unclear model architectures, data privacy risks, and expensive inference overheads. To address the limitations, we introduce CodeS, a series of pre-trained language models with parameters ranging from 1B to 15B, specifically designed for the text-to-SQL task. CodeS is a fully open-source language model, which achieves superior accuracy with much smaller parameter sizes. This paper studies the research challenges in building CodeS. To enhance the SQL generation abilities of CodeS, we adopt an incremental pre-training approach using a specifically curated SQL-centric corpus. Based on this, we address the challenges of schema linking and rapid domain adaptation through strategic prompt construction and a bi-directional data augmentation technique. We conduct comprehensive evaluations on multiple datasets, including the widely used Spider benchmark, the newly released BIRD benchmark, robustness-diagnostic benchmarks such as Spider-DK, Spider-Syn, Spider-Realistic, and Dr.Spider, as well as two real-world datasets created for financial and academic applications. The experimental results show that our CodeS achieves new SOTA accuracy and robustness on nearly all challenging text-to-SQL benchmarks.

Edge artificial intelligence applications impose rigorous demands on local hardware to improve throughput and energy efficiency. Computing-in-memory (CIM) architectures provide high parallel and energy-efficient solutions to accelerate the multiply-and-accumulate (MAC) operations in neural networks (NNs). While SRAM-based charge-domain CIM is achieving thousands of TOPS/W energy efficiency, it encounters limitations when dealing with full NN model deployments where both activations and weights are signed. This paper proposes an SRAM-based signed batch normalization (BN) CIM macro for supporting efficient bitwise sparse MAC computation with signed operands and BN operations in deep neural networks. The key features of this macro encompass: 1) a multibit weight unit for the optimization of bitstream sparsity and the sign bit computation, 2) a 2b-serial input configuration to increase throughput and the ADC energy amortization, and 3) a quantization-hardware co-design for the BN implementation. Measurement results show that the proposed 28 nm 64 Kb CIM macro achieves 16.38 TOPS throughput and 4.55 POPS/W energy efficiency, both normalized to 1b operands. The test accuracy of CIFAR10 is 92%, based on the ResNet18 model with co-design BN implementation at signed-8b precision activations and weights.

End-to-end stochastic computing (SC) enables fault-tolerant and area-efficient neural acceleration by conducting non-linear addition, including accumulation and activation functions, in SC bitstreams. However, existing non-linear adder designs suffer from a high hardware cost, accounting for a major portion of the datapath power and area, and may also have limited computation accuracy and flexibility. In this paper, we propose an accurate yet efficient non-linear adder design. We analyze the redundancy in existing designs and propose a parameterized approximate non-linear adder design space. By systematic design space exploration, we develop non-linear adders that are significantly more efficient than existing designs with negligible computation error. We further propose a spatial-temporal architecture to improve the design flexibility and efficiency for a wide range of network sizes. To support state-of-the-art networks, e.g., ResNet18, we demonstrate that our design can reduce the datapath area by 2.16× compared with the baseline designs. Our design can also reduce the area-delay product (ADP) of the non-linear adder by 4.13× and 23.29× for large and small convolution layers in ResNet18, respectively.

Stochastic computing (SC) emerges as a fault-tolerant and area-efficient computing paradigm for neural acceleration. However, existing SC accelerators suffer from an intrinsic trade-off between inference accuracy and efficiency: accurate SC re-quires high precision computation but suffers from an exponential increase of bitstream length and inference latency. In this paper, we discover the high precision residual as a key remedy and propose to combine a low precision datapath with a high precision residual to improve inference accuracy with minimum efficiency overhead. We also propose to fuse batch normalization with the activation function to further improve the inference efficiency. The effectiveness of our proposed method is verified on a recently proposed SC accelerator. With extensive results, we show that our proposed SC-friendly network achieves 9.43% accuracy im-provements compared to the baseline low precision networks with only 1.3% area-delay product (ADP) increase. We further show$\boldsymbol{3.01\times}$ADP reduction compared to the baseline SC accelerator with almost iso-accuracy.

With the rapid advancements of deep learning in recent years, hardware accelerators are continuously deployed in more and more safety-critical applications such as autonomous driving and robotics. While the accelerators are usually fabricated with advanced technology nodes for high performance and energy efficiency, they are also more prone to timing errors under process, voltage, temperature, and aging (PVTA) variations. By revisiting the physical sources of timing errors, we show that most of the timing errors in the accelerator are caused by a specific subset of input patterns, defined as critical input patterns. To improve the timing error resilience of the accelerator, in this paper, we propose READ, a reliability-enhanced acceler-tor dataflow optimization technique that can effectively reduce timing errors. READ reduces the occurrence of critical input patterns by exploring the optimal computing sequence when mapping a trained deep neural network to accelerators. READ only changes the order of multiply-accumulate operations in a convolution, which introduces negligible hardware overhead and no impact on accuracy. The experimental results on VGG and ResNet demonstrate on average$\mathbf{7.8}\times$timing error rate (TER) reduction and up to$\mathbf{37.9}\times$TER reduction for certain layers. The results also show that READ enables the accelerator to maintain accuracy over a wide range of PVTA variations, making it a promising approach for robust deep-learning design.