This paper presents a comprehensive evaluation of microscaling (MX) quantization in the pre-training of large language models (LLMs), investigating its potential to enhance the computation and memory efficiencies. We systematically examine the effects of key design parameters - including data types, rounding modes, scaling strategies, granularity, and organization - on numerical accuracy and training stability. Our extensive experimental study on Llama3 models reveals critical insights into the challenges of 4-bit training for LLMs and identifies optimal configurations with mixed precisions of 4-bit and 6-bit MX formats that significantly enhance training quality, bridging the gap with higher-precision formats. This research provides valuable guidance on the benefits and limitations of MX quantization, laying the groundwork for future innovations in low-precision LLM training.

This paper introduces a new class of space-time codes that achieve coding gain without a trellis or any form of inter-block dependency. The construction of the new codes starts from an existing (parent) space-time block code (STBC). Then by increasing the constellation size followed by expurgation of the expanded codebook, a better code is obtained at the original transmission rate. This method can be applied to a wide variety of space-time block codes, including orthogonal codes and quasi-orthogonal codes. A multi-stage design algorithm is presented, and for orthogonal parent codes, an efficient decoding algorithm is developed, and its decoding complexity is analyzed. Despite altering the regular structure of the orthogonal code, the decoding complexity is only affected by a constant factor.

Concatenation of orthogonal space-time block codes (OSTBC) with an outer trellis has led to simple and powerful codes, known as superorthogonal codes or space-time block trellis-coded modulation. In this letter, we generalize these codes by finding new code supersets and corresponding set partitioning, resulting in improved coding gain. We provide design guidelines for the labeling of the generalized code trellises and demonstrate the gains by several example designs for two and four transmit antennas

We propose a new class of block codes that outperforms known space-time block codes at low rates. The new codes are designed by using appropriate rotations and set partitioning on two quasi-orthogonal codes, and combining subsets of their codewords. Using these techniques we are able to obtain higher coding gain at a given rate and improve performance. Simulations confirm the advantages of this code compared to other codes operating at the same rate and signal-to-noise ratio (SNR). We also provide an efficient maximum likelihood (ML) decoding algorithm for the new code

We propose a new class of block codes that outperforms known space-time block codes at low rates. The new codes are designed by starting with a quasiorthogonal structure, and then making certain modifications to increase the coding gain distance. By using appropriate rotations and set partitions for two quasi-orthogonal codes, and combining subsets of their codewords, we are able to obtain higher coding gain distance at a given rate, and thus improve performance. Simulations confirm the advantages of this code compared to other codes operating at the same rate and SNR. We also provide an efficient ML decoding algorithm for the new code.

This paper presents a method for increasing the coding gain of all varieties of space-time block codes (STBC), without using a trellis or introducing dependency between successive transmission blocks, using ideas from block-coded modulation (BCM). For a given STBC, we first increase the constellation size, then prune the codewords of the expanded codebook according to distance criteria, so that we arrive at the original transmission rate. We show that it is possible to improve the performance of a wide variety of space-time signalings, including orthogonal codes, quasi-orthogonal codes. An algorithm for the code design is presented. In the case of orthogonal codes, a decoding algorithm for the modified orthogonal codes is presented, showing that despite altering the regular structure of the orthogonal code, the complexity of decoding is only affected by a small constant. The same principle also applies to a wide variety of codes such as LD and TAST codes, whose design examples are under construction at the time of writing of this manuscript

It is well-known that diversity, despite being widely used as a design criterion, may not be enough to ensure good performance of a wireless system, because the diversity factor may "kick in" at unrealistically high values of SNR. This paper proposes a new class of layered space-time codes with a new design criterion that works well in moderate SNR's. Specifically, we propose to relax some of the constraints of threaded algebraic space-time (TAST) codes, leading to a class of codes with better error performance, which we call relaxed threaded space time (RTST) codes. We also propose a modified design criterion, the average union bound (AUB), which ensures good performance at medium SNR. For a 2 times 2 system operating at 6 b/s/Hz, improvements of more than 1.5 dB have been observed

We develop algorithms to efficiently build trellises for various full-rate MIMO codes. By full-rate, we refer to codes for multiple antenna systems whose rate scales with the minimum of the number of transmit and receive antennas, e.g., BLAST and the linear dispersion codes of Hassibi and Hochwald. This is in part inspired by the so-called super-orthogonal codes, which build efficient trellises on orthogonal block space-time codes (e.g. the Alamouti code). Unfortunately that approach cannot be directly transferred to a code with insufficient structure, because set partitioning over an irregular set, such as the one represented by an arbitrary space-time code, is not straightforward. The central contribution of this paper is an efficient set partitioning algorithm for an arbitrary set. We then built trellises for the resulting set partitions and demonstrate via simulations the gains obtained by such trellis codes.

Orthogonal space-time block codes (OSTBC) enjoy simple decoding, but have limited coding gain, if any. By concatenating OSTBC with an outer trellis, simple and powerful codes have been constructed, known as super-orthogonal codes or STB-TCM. In this work, we generalize these codes by exploring new code supersets, through mappings that do not induce any instantaneous modulation constellation expansion. By finding new mappings and establishing the properties of the resulting set partitions, we provide design guidelines for the labeling of the generalized code trellises. Simulations demonstrate significant coding gains resulting from our codes.