Transformer models achieve state-of-the-art accuracy on natural language processing (NLP) and vision tasks, but demand significant computation and memory resources, which makes it difficult to perform inference and training (fine-tuning) on edge accelerators. Quantization to lower precision data types is a promising way to reduce computation and memory resources. Prior work has employed 8-bit integer (int8) quantization for Transformer inference, but int8 lacks the precision and range required for training. 8-bit floating-point (FP8) quantization has been used for Transformer training, but prior work only quantizes the inputs to matrix multiplications and leaves the rest of the operations in high precision.

CRC-aided belief propagation list (CBPL) is a low latency, high throughput decoder for polar codes. In this work, it is shown that CBPL decoding of polar codes over the additive white Gaussian noise channel can be substantially improved using ordered statistics post-processing that only requires a very low reprocessing order (e.g., order one). We present efficient implementations of the ordered statistics post-processing with a trade-off between computational complexity and error rate.

A new algorithm for efficient exact maximum likelihood decoding of polar codes (which may be CRC augmented), transmitted over the binary erasure channel, is presented. The algorithm applies a matrix triangulation process on a sparse polar code parity check matrix, followed by solving a small size linear system over GF(2). To implement the matrix triangulation, we apply belief propagation decoding type operations. We also indicate how this decoder can be implemented in parallel for low latency decoding. Numerical simulations are used to evaluate the performance and computational complexity of the new algorithm.

The problem of communicating over an additive white Gaussian noise channel with feedback, using low precision arithmetic, is considered. The Schalkwijk-Kailath (SK) scheme is known to achieve an error probability that decays double exponentially in the number of interaction rounds, for any rate below channel capacity. However, SK is also known to suffer from numerical issues. In this work we propose a new, modified scheme termed Zoom-in SK (ZSK), which breaks the SK protocol into several stages. Each stage comprises several SK iterations followed by a synchronized zoom step. The zoom-in allows the receiver and transmitter to keep the scheme's parameters relatively large such that low precision arithmetic can be used. We prove that the new scheme achieves approximately the same error probability as SK while not suffering from numerical issues. We further verify our results in simulation and compare ZSK to the original SK scheme.