Transformer-based large language models are a memory-bound model whose operation is based on a large amount of data that are marginally reused. Thus, the data movement between a host and accelerator likely dictates the total wall-clock time. Layer normalization is one of the key workloads in the transformer model, following each of multi-head attention and feed-forward network blocks. To reduce data movement, layer normalization needs to be performed on the same chip as the matrix-matrix multiplication engine. To this end, we introduce an iterative L2-normalization method for 1D input (IterL2Norm), ensuring fast convergence to the steady-state solution within five iteration steps and high precision, outperforming the fast inverse square root algorithm in six out of nine cases for FP32 and five out of nine for BFloat16 across the embedding lengths used in the OPT models. Implemented in 32/28nm CMOS, the IterL2Norm macro normalizes d-dimensional vectors, where 64 ≤$d$≤ 1024, with a latency of 116–227 cycles at 100MHz/l.05V.

Manycore digital neuromorphic event processors execute ad hoc event routing between spiking neurons distributed across multiple cores. Due to limited hardware resources, such as on-chip memory capacity, only a limited number of neurons and their fan-in weights can be accommodated per core. This challenge is particularly significant for convolutional layers, where placing neurons from the same layer in different cores hinders weight reuse, as the same weight must be duplicated across cores. To address this, we propose an optimal mapping method for spiking units across multiple cores, considering hardware resource constraints. This method is based on the discrete Lagrange Multiplier Method, which uses a total memory usage as an objective function alongside constraint functions (memory usage per core). Our results show that this method achieves optimal spiking unit distributions with high core memory utilization (> 70%) for the reduced ResNet models.

We present a method to emulate a leaky integrate-and-fire (LIF) model in a field-programmable gate array (FPGA) in a hardware-efficient manner. The simplified spike-response model (SRM0) is chosen as an LIF model. For the hardware-efficient implementation of SRM0, we adopt the template-scaling-based exponential function approximation (TS-EFA). This method allows high precision and low latency exponential function approximations with the efficient use of hardware resources. We subsequently propose an algorithm for SRM0, which leverages the advantage of TS-EFA. An implementation of 512 neurons conforming to SRM0in an FPGA highlights (i) high precision of SRM0emulation (mean squared error of membrane potential approximation: 4×10-12- 1×10-10), (ii) low latency (eight clock cycles), and (iii) high efficiency in hardware usage (only 125b memory per neuron).