Error correcting codes (ECCs) are indispensable for reliable transmission in communication systems. Recent advancements in deep learning have catalyzed the exploration of ECC decoders based on neural networks. Among these, transformer-based neural decoders have achieved state-of-the-art decoding performance. In this paper, we propose a novel Cross-Attention Message-Passing Transformer (CrossMPT), which shares key operational principles with conventional message-passing decoders. While conventional transformer-based decoders employ a self-attention mechanism without distinguishing between magnitude and syndrome embeddings, CrossMPT updates these two types of embeddings separately and iteratively via two masked cross-attention blocks. The mask matrices are determined by the code's parity-check matrix, which explicitly captures and removes irrelevant relationships between the magnitude and syndrome embeddings. Our experimental results show that CrossMPT significantly outperforms existing neural network-based decoders for various code classes. Notably, CrossMPT achieves this decoding performance improvement while significantly reducing memory usage, computational complexity, inference time, and training time.

Ensuring extremely high reliability in channel coding is essential for 6G networks. The next-generation of ultra-reliable and low-latency communications (xURLLC) scenario within 6G networks requires frame error rate (FER) below 10-9. However, low-density parity-check (LDPC) codes, the standard in 5G new radio (NR), encounter a challenge known as the error floor phenomenon, which hinders to achieve such low frame error rates. To tackle this problem, we introduce an innovative solution: boosted neural min-sum (NMS) decoder. This decoder operates identically to conventional NMS decoders, but is trained by novel training methods including: i) boosting learning with uncorrected vectors, ii) block-wise training schedule to address the vanishing gradient issue, iii) dynamic weight sharing to minimize the number of trainable weights, iv) transfer learning to reduce the required sample count, and v) data augmentation to expedite the sampling process. Leveraging these training strategies, the boosted NMS decoder achieves the state-of-the art performance in reducing the error floor as well as superior waterfall performance. Remarkably, we fulfill the 6G xURLLC requirement for 5G LDPC codes without a severe error floor. Additionally, the boosted NMS decoder, once its weights are trained, can perform decoding without additional modules, making it highly practical for immediate application. The source code is available athttps://github.com/ghy1228/LDPC_Error_Floor.

With the broadening applications of deep learning, neural decoders have emerged as a key research focus, specifically aimed at improving the decoding performance of conventional decoding algorithms. In particular, error correction code transformer (ECCT), which utilizes the transformer architecture, has achieved state-of-the-art performance among neural network-based decoders. We present three technical contributions to significantly enhance the performance of ECCT. First, we propose a novel transformer architecture of ECCT, termed themultiple-masks ECCT (MM ECCT). We employ multiple masked self-attention blocks with different mask matrices in a parallel manner to learn diverse relationships among the codeword bits. Second, we discover that constructing mask matrices based on systematic parity check matrices (PCMs) can make the attention mapssparse, which not only enhances the decoding performance but also reduces computational complexity. Finally, we propose using complementary mask matrices derived from cyclic permutations of the systematic PCM. These complementary mask matrices are specifically designed to enhance the decoding of cyclic codes. Our extensive simulation results show that the proposed MM ECCT architecture with carefully designed mask matrices outperforms the original ECCT by a large margin, achieving state-of-the-art decoding performance among neural decoders. The source code is available at https://github.com/iil-postech/mm-ecct.

Low-density parity-check (LDPC) codes have been successfully commercialized in communication systems due to their strong error correction capabilities and simple decoding process. However, the error-floor phenomenon of LDPC codes, in which the error rate stops decreasing rapidly at a certain level, presents challenges for achieving extremely low error rates and deploying LDPC codes in scenarios demanding ultra-high reliability. In this work, we propose training methods for neural min-sum (NMS) decoders to eliminate the error-floor effect. First, by leveraging the boosting learning technique of ensemble networks, we divide the decoding network into two neural decoders and train the post decoder to be specialized for uncorrected words that the first decoder fails to correct. Secondly, to address the vanishing gradient issue in training, we introduce a block-wise training schedule that locally trains a block of weights while retraining the preceding block. Lastly, we show that assigning different weights to unsatisfied check nodes effectively lowers the error-floor with a minimal number of weights. By applying these training methods to standard LDPC codes, we achieve the best error-floor performance compared to other decoding methods. The proposed NMS decoder, optimized solely through novel training methods without additional modules, can be integrated into existing LDPC decoders without incurring extra hardware costs. The source code is available at https://github.com/ghy1228/LDPC_Error_Floor.

In this paper, we propose an optimization method for protograph-based spatially coupled low-density parity-check (SC-LDPC) codes under window decoding (WD). Previous works on constructing SC-LDPC codes for WD typically focused on optimizing asymptotic performance metrics such as the WD threshold. However, in this paper, it is observed that the WD threshold is not an appropriate metric to sufficiently explain the finite-length behavior of SC-LDPC codes under WD. Thus, we propose a new performance metric, called the window mean parameter, based on a scaling analysis to capture the WD performance more accurately and formulate a code optimization algorithm that optimizes the proposed performance metric. Since the proposed metric depends on the window size, the optimization algorithm can provide a code family of SC-LDPC codes optimized for various target window sizes. Simulation results confirm that the improvement in the proposed metric leads to a finite-length performance improvement, resulting in one to two orders of the frame error rate gain over the conventional SC-LDPC codes for a wide range of window sizes. Furthermore, we investigate structural characteristics of the proposed codes to provide a supplementary explanation for the performance improvement, which also promotes a better understanding of SC-LDPC codes for WD.

In research on spatially-coupled low-density parity-check (SC-LDPC) codes, rate-loss of SC-LDPC codes is one of the main issues to be addressed. One way to mitigate the rate-loss is to attach additional variable nodes with an irregular degree distribution, where the degree distribution is optimized with a constraint that the belief propagation (BP) threshold should not be degraded by attaching variable nodes. However, it is observed that the degree distribution obtained with the BP threshold constraint induces degradation of the finite-length performance. In order to address the problem, we propose new optimization methods to attach additional variable nodes while minimizing performance degradation. The proposed optimization methods are based on several design techniques including the scaling law, local threshold, expected graph evolution, differential evolution algorithms, the use of a protograph structure, and puncturing codewords. Using the optimized structure for additional variable nodes, the rate-loss of SC-LDPC codes can be reduced by more than 53% without sacrificing the finite-length performance. It is also shown that the rate-loss mitigation can be translated into a performance improvement if the proposed and the conventional SC-LDPC codes are compared at the same code rate.

Irregular quasi-cyclic (QC) low-density parity-check (LDPC) codes with the block dual-diagonal (BDD) parity structure are widely adopted in many communication standards because the BDD structure supports an efficient encoding and many degree-2 variable nodes inside are adequate for the construction of mid- to high-rate codes. However, we observe that low-rate irregular QC LDPC codes with the BDD parity structure inherently contain too many degree-2 variable nodes and suffer from error floors in high signal-to-noise ratio (SNR) region. In this paper, a generalized BDD structure including double-weight circulants as well as circulant permutation matrices is proposed for low-rate irregular QC LDPC codes with low error floors which is achieved with a little bit giving up error performance in the waterfall region. When constructing the parity part of a code with the generalized BDD structure, the portion of double-weight circulants is variable so that the resulting LDPC code can achieve a desired degree distribution including degrees 2 and 3 while supporting the efficient encoding. We show that low-rate QC LDPC codes constructed with the proposed BDD structure have better theoretical properties and lower error floor than those with the conventional BDD structure.

In this paper, we propose new design algorithms of irregular spatially-coupled low-density parity-check (SC-LDPC) codes with non-uniform degree distributions using linear programming (LP). In general, irregular SC-LDPC codes with non-uniform degree distributions are difficult to design with low complexity because their density evolution equations are multi-dimensional. To overcome this problem, proposed design algorithms are based on three main ideas: a local design of degree distributions, pre-computation of the input/output message relationship, and selection of a proper objective function. These ideas make it possible to design degree distributions of irregular SC-LDPC codes by solving low-complexity LP problems over the binary erasure channel (BEC). It is shown that the proposed irregular SC-LDPC codes designed by the proposed algorithms are superior to regular SC-LDPC codes in terms of both asymptotic and finite-length performances over the BEC. We also confirm that the proposed irregular SC-LDPC code achieves better performance compared with an optimized irregular block LDPC code in the same blocklength, which implies that the proposed design algorithms also provide a new way to construct capacity-approaching block LDPC codes.

The outstanding decoding performance of spatially-coupled low-density parity-check (SC-LDPC) codes comes from wave-like propagation of reliable messages. The reliable messages are triggered by shortened (known) variable nodes in some consecutive reliable positions. However, at the cost of the improvement, shortened variable nodes cause rate-loss of SC-LDPC codes. To reduce the rate-loss, additional variable nodes (so called reliable variable nodes) can be added to the reliable positions instead of shortened variable nodes. Density evolution (DE) is an efficient method to design degree distribution of the reliable variable nodes. However, degree distributions obtained by DE show degraded performance in finite-length code performance. In this paper, we generalize the expected graph evolution and use the analysis tool in optimizing degree distribution which shows the minimum rate-loss without finite-length performance degradation. From the well-designed degree distribution, rate-loss reduction by 60% can be achieved without finite-length performance degradation.