New coding schemes based on generalized concatenated codes are proposed for emerging nonvolative memory technologies. Key requirements are high code rate, low latency, high throughput and the ability to correct chipkill failures while sustaining high data reliability despite raw bit error rates up to 10-3. New concatenated codes based on Reed-Solomon codes have been designed for payload sizes of 512B, 1kB, and 2kB; they have high rates above 0.8 and a high data reliability with a decoder target BER of 10-15. An FPGA-based implementation of the decoder validates the low latency and high throughput: for an operating clock frequency of 250MHz, the decoding latency is 236ns and a 9.3GB/s throughput is achieved.

The Earth Mover’s Distance (EMD) is a state-of-the art metric for comparing discrete probability distributions, but its high distinguishability comes at a high cost in computational complexity. Even though linear-complexity approximation algorithms have been proposed to improve its scalability, these algorithms are either limited to vector spaces with only a few dimensions or they become ineffective when the degree of overlap between the probability distributions is high. We propose novel approximation algorithms that overcome both of these limitations, yet still achieve linear time complexity. All our algorithms are data parallel, and therefore, we can take advantage of massively parallel computing engines, such as Graphics Processing Units (GPUs). On the popular text-based 20 Newsgroups dataset, the new algorithms are four orders of magnitude faster than a multi-threaded CPU implementation of Word Mover’s Distance and match its search accuracy. On MNIST images, the new algorithms are four orders of magnitude faster than Cuturi’s GPU implementation of the Sinkhorn’s algorithm while offering a slightly higher search accuracy.

Next-generation memory (NGM) technologies present a major opportunity but also a significant challenge, due to their intricate reliability issues. In particular, multilevel-cell (MLC) storage is highly desirable for increasing storage capacity and lowering total cost-per-bit. In phase-change memory (PCM), MLC storage is hampered by sensitivity to temperature variations and resistance drift. A novel drift-invariant detection (DID) scheme that estimates variable read thresholds based on ordered statistics and clustering of the soft read-back signals from a small block of 32 cells has been developed and implemented in hardware to improve reliability and prolong data retention. A low-complexity implementation of the DID on a FPGA platform comprises 20'000 LUTs and 6'000 flip-flops and has a latency of 90ns. We present results from an extensive performance verification that ascertains highly reliable data retrieval up to 13 orders of magnitude in time after programming. Such elevated reliability is necessary for the most anticipated application of NGM, namely persistent far-memory, where the NGM is used as a large memory pool, possibly together with DRAM.

The bit-error-rate (BER) performance of 3-dimensional (3D) product codes under iterative bounded-distance decoding of the component codes is considered and a framework for analyzing the BER-performance is presented. The performance analysis of iterative decoding is based on a graphical model of the underlying 3D product code, which is a tripartite 3-uniform hypergraph. The BER performance for 3D product codes shows a threshold behavior. The thresholds can be approximated by an exit-chart-like technique from the parameters of the component codes. In the case of a symmetric product code, for which all three component codes are based on the same lineart-error correcting code, asymptotically for large code lengths, the BER-threshold is determined by the threshold for the appearance of ak-core withk=t+ 1 in the graphical model.

In this paper, we develop a framework for evaluating the symmetric capacity of multilevel-cell (MLC) NAND flash devices while making very few assumptions regarding the underlying device physics. A set of recursive equations are derived that allow one to measure the symmetric capacity for any given page in a flash device using simple conditional statistics that can be extracted experimentally. Using data captured from two different 1y nm MLC devices, we demonstrate that the symmetric capacity of a flash page not only depends on the amount of program/erase cycling and data retention stress that has accumulated, but also on the position of the page within the flash block. We then study the effect on symmetric capacity of using optimized read-back schemes (both hard and soft) and show that while there is significant benefit, not all pages in the block are improved by the same amount. Finally, we show that it is possible to design error correction architectures that harness the inherent variation of symmetric capacity within a flash block to dramatically extend the program/erase cycling endurance of flash-based storage systems.

An extensive effort is being undertaken by the flash community to develop signal processing and error-correction coding schemes that make use of soft information. Using experimental data from a state-of-the-art MLC flash device we demonstrate that the theoretical endurance improvement that such schemes can bring is limited. To investigate further, we develop a parametric channel model that takes into account the effects of cell-to-cell interference and demonstrate that it is the presence of programming errors in the channel that restricts the potential endurance enhancement that soft information can offer.

Recently, a new type of product-like codes, known as half-product codes, have been studied for OTN applications. Motivated by these codes, new classes of symmetry-invariant subproduct codes are proposed and investigated under iterative hard-decision decoding. A subset of the new class of quarter product codes has lower error floors than comparable half-product codes in terms of length, rate and performance.

NAND flash memory is not only the ubiquitous storage medium in consumer applications but has also started to appear in enterprise storage systems as well. MLC and TLC flash technology made it possible to store multiple bits in the same silicon area as SLC, thus reducing the cost per amount of data stored. However, at current sub-20nm technology nodes, MLC flash devices fail to provide the levels of raw reliability, mainly cycling endurance, that are required by typical enterprise applications. Advanced signal processing and coding schemes are needed to improve the flash bit error rate and thus elevate the device reliability to the desired level. In this article, we report on the use of adaptive voltage thresholds and cell-to-cell interference cancellation in the read operation of NAND flash devices. We discuss how the optimal read voltage thresholds can be determined and assess the benefit of cancelling cell-to-cell interference in terms of cycling endurance, data retention, and resilience to read disturb.

The proliferation of NAND flash memory in consumer devices has driven their aggressive cost reduction by continuous scaling to smaller technology nodes. However, this relentless cost per capacity improvement has diminished the reliability of flash memory to a degree that advanced signal processing and error correction are needed to enhance signal integrity in current flash-based systems. Accurate models of flash readback signals are necessary to properly design such advanced signal enhancement schemes. We propose a new parametric model of the flash readback signal based on fitting threshold voltage distributions from NAND flash devices. We show accurate fitting results for flash devices cycled up to 10 times longer than their nominal endurance specification, and provide simple expressions of the model parameters as a function of program/erase cycles. Finally, we also demonstrate that the proposed model can be used to capture effects such as programming errors, that occur in over-stressed flash devices.

NAND Flash memory is not only the ubiquitous storage medium in consumer applications, but has also started to appear in enterprise storage systems as well. MLC and TLC Flash technology made it possible to store multiple bits in the same silicon area as SLC, thus reducing the cost per amount of data stored. However, at current sub-20nm technology nodes, MLC Flash devices fail to provide the levels of raw reliability, mainly cycling endurance, that are required by typical enterprise applications. Advanced signal-processing and coding schemes are needed to improve the Flash bit error rate and thus elevate the device reliability to the desired level. In this paper, we report on the use of adaptive voltage thresholds in the read operation of NAND Flash devices. We discuss how the optimal read voltage thresholds can be determined, and assess the benefit of adapting the read voltage thresholds in terms of cycling endurance, data retention and resilience to read disturb.

We consider high-rate modulation codes satisfying tight global and interleave (G, I) constraints for magnetic storage systems. We use a novel sliding-window substitution coding technique to improve on known long capacity-efficient (G = 2γ, I = γ) codes. This coding technique maps one-sided (G = 2γ, I = γ)-constrained sequences into one-sided sequences satisfying a substantially tighter G constraint and a slightly relaxed I constraint. Sliding-window substitution encoding in conjunction with enumerative encoding provides high-rate capacity-efficient codes that are relevant for practical magnetic storage systems.

A number of modern storage technologies, when written to, exhibit substantial variability in the outcome of a write action. It is possible to mitigate the effect of the write uncertainty through the use of a feedback loop that rewrites the memory whenever judged necessary, in effect reshaping the write noise. This scheme highlights a tradeoff between the storage capacity of the memory and the cost of writing to it, measured for example in the number of rewrites. We have developed the model of a rewritable channel to provide an explicit form for this tradeoff and study other performance characteristics of such memories. In this paper, we describe some initial results on the information-theoretic analysis of the rewritable channel. We first consider the problem of determining the capacity of this channel with input cost constraints, and obtain a variety of results from which we extract insights that we believe are of value to memory designers. Our results include an upper bound on capacity of the form log (Γκ), where Γ is a constant that can be easily calculated from the channel's statistics and κ is an average cost parameter. We also provide a lower bound on capacity with a similar form. We analyze the particular case of uniform write noise in detail, obtaining a closed form expression for the capacity-cost tradeoff for all possible cost parameters. We explore this formula from the capacity per unit cost perspective and establish that in order to achieve optimal energy and memory-wear per bit, it is sometimes strictly better to take advantage of the rewriting capability as opposed to writing only once; this observation has significant practical implications. We also include a discussion of the relevance of our work to real emerging memory technologies.

We present a closed form expression for the capacity of the uniform noise rewritable channel with average write cost and a constraint on the input range. We show the existence of a critical cost κ0such that for all costs κ ≥ κ0, the capacity/cost tradeoff is given by an offset added to the logarithm of the cost. Assuming κ0> 1, for 1 ≤ κ0the capacity/cost tradeoff grows faster than a logarithm.

We consider storage channels that admit optional reading and rewriting of the content at a given cost. This is a general class of channels that models many nonvolatile memories. We present recent results on such rewritable channels with constraints on both the maximum and the average number of atomic rewrite iterations. We derive a general lower capacity bound for rewritable storage channels impaired by additive noise. For the special case of uniform noise, we present tight upper and lower capacity bounds and suggest some capacity-achieving coding techniques.

We present some recent results on rewritable channels, that is, storage channels that admit optional reading and rewriting of the content at a given cost. This is a general class of channels that models many nonvolatile memories. We focus on the storage capacity of rewritable channels affected by data-dependent noise. We prove tight upper and lower bounds on the storage capacity of a simple yet significant channel model and suggest some simple capacity-achieving coding techniques. Lower bounds on the storage capacity of Gaussian rewritable channels with data-dependent noise are also shown.

We study memories capable of storing multiple bits per memory cell, with the property that certain state transitions “wear” the cell. We introduce a model that is relevant for Phase Change Memory, a promising emerging nonvolatile memory technology that exhibits limitations in the number of particular write actions that one may apply to a cell before rendering it unusable. We exploit the theory of Write Efficient Memories to derive a closed form expression for the storage capacity/lifetime fundamental tradeoff for this model. We then present families of codes specialized to distinct ranges for the target lifetimes, covering the full range from moderate redundancy to an arbitrarily large lifetime increase. These codes have low implementation complexity and remarkably good performance; for example in an 8 level cell we can increase the lifetime of a memory by a factor of ten while sacrificing only 2/3 of the uncoded storage capacity of the memory.

A new class of maximum-transition-run (MTR) block codes is presented which is based on a novel enumerative encoding scheme to construct codes with predetermined j and k constraints. These MTR codes are a generalization of Fibonacci codes, and they share a similar optimality property: For a given length N, they contain the maximum number of length-N codewords satisfying the (j, k) constraint. Furthermore, by using a slight modification of the code construction, one obtains practical high-rate modulation codes with limited error propagation.

Reverse concatenation (RC) architectures, which recently have been deployed in hard-disk-drive (HDD) products, offer crucial advantages in coding such as (i) avoiding error propagation through the modulation decoder, (ii) allowing the use of efficient high-rate modulation codes, and (iii) passing of soft information from the detector to the decoder, which facilitates parity-post processing and iterative coding schemes. In HDDs, error-correcting codes essentially consist of a single high-rate Reed-Solomon code, whereas in tape recording, large product codes are used that require a new RC architecture. Such a novel RC architecture for product codes is presented and illustrated by an example based on the linear tape open standard, generation 4 (LTO-4). Compared with the rate-16/17 modulation code of the LTO-4 standard, the proposed RC scheme has a modulation scheme of rate 0.9951, i.e., achieves 5.7% improvement in rate while maintaining the same interleaved I = 11 modulation constraint, but at the cost of a slight weakening of the G-constraint.

A reverse concatenation scheme based on an efficient modulation encoder satisfying non-uniform constraints, a systematic Reed-Solomon encoder and a partial symbol interleaver is presented. This architecture achieves very tight modulation constraints and minimizes error propagation and rate loss. The modulation constraints considered are of the same type as the constraints that have been used in generalized partial-response maximum-likelihood (PRML) detection systems. A class of codes that is based on serial concatenation of a prefix-constrained code and two interleaved enumerative codes with non-uniform constraints is proposed. Specific rate-199/200 PRML(G, I) codes are constructed.

To mitigate transition noise in magnetic recording is to reduce the number of transitions that are written onto the media. This leads one to consider an average transition density (ATD) constraint. This new constraint reduces the ATD of sequences at the channel input and can be imposed in addition to other constraints. In this paper ATD constraint is imposed which is based on assigning a particular cost function to a finite-state transition diagram (FSTD) that presents a constrained system of binary NRZI sequences. At a normalized linear user density the performance of an unconstrained system satisfying an ATD constraint was simulated using a microtrack longitudinal (Lorentzian) recording model corrupted by transition noise and electronics noise.

The basic algebraic structure theory of convolutional codes and their trellises is developed simultaneously for codes over groups, rings, and fields. The first part, which covers fundamental notions such as minimality and observability, is semi-tutorial in that most definitions are already standard (within the modern behavioral theory), as are some of the formally stated results. However, some of the pivotal results-emphasizing the role of observability as the basic well-behavedness condition for codes-are new, and several previous results are given simplified proofs. The usefulness of the behavioral approach even for convolutional codes over fields is demonstrated by a new minimality test for encoders as well as by the straightforward derivation of some known minimality criteria for generator matrices from the basic minimality criteria for group trellises. The second part of the paper deals with issues that are specific to codes over rings and groups. The main result is a concise characterization-the first such-of those groups that can appear as the branch group of any group trellis. It is further shown how such groups are "presented" by shift registers. A new large class of noncommutative convolutional codes is also given.

Slepian-type group codes generated by finite Coxeter groups are considered. The resulting class of group codes is a generalization of the well-known permutation modulation codes of Slepian (1965), it is shown that a restricted initial-point problem for these codes has a canonical solution that can easily be computed. This allows one to enumerate all optimal group codes in this restricted sense and essentially solves the initial-point problem for all finite reflection groups. Formulas for the cardinality and the minimum distance of such codes are given. The new optimal group codes from exceptional reflection groups that are obtained achieve high rates and have excellent distance properties. The decoding regions for maximum-likelihood (ML) decoding are explicitly characterized and an efficient ML-decoding algorithm is presented. This algorithm relies on an extension of Slepian's decoding of permutation modulation and has similar low complexity,.