Reliable communication over bandlimited and nonlinear channels usually requires equalization to simplify receiver processing. Equalizers that perform joint detection and decoding (JDD) achieve the highest information rates but are often too complex to implement. To address this challenge, model-based neural network (NN) equalizers that perform successive interference cancellation (SIC) are shown to approach JDD information rates for bandlimited channels with a memoryless nonlinearity and additive white Gaussian noise. The NNs are chosen to have a periodically time-varying and recurrent structure that imitates the forward-backward algorithm (FBA) in every SIC stage. Simulations for short-haul fiber-optic links with square-law detection show that NN-SIC nearly doubles current spectral efficiencies, and bipolar or complex-valued modulations achieve energy gains of up to 3 dB compared to state-of-the-art intensity modulation. Moreover, NN-SIC is considerably less complex than equalizers that perform JDD, mismatched FBA processing, and Gibbs sampling.

Neural networks (NNs) inspired by the forward-backward algorithm (FBA) are used as equalizers for bandlimited channels with a memoryless nonlinearity. The NN-equalizers are combined with successive interference cancellation (SIC) to approach the information rates of joint detection and decoding (JDD) with considerably less complexity than JDD and other existing equalizers. Simulations for short-haul optical fiber links with square-law detection illustrate the gains.

The last decade has witnessed the breakthrough of deep neural networks (DNNs) in many fields. With the increasing depth of DNNs, hundreds of millions of multiply-and-accumulate (MAC) operations need to be executed. To accelerate such operations efficiently, analog in-memory computing platforms based on emerging devices, e.g., resistive RAM (RRAM), have been introduced. These acceleration platforms rely on analog properties of the devices and thus suffer from process variations. Consequently, weights in neural networks configured into these platforms can deviate from the nominal trained values, which may lead to feature errors and a significant degradation of the inference accuracy. Besides, additional HW aspects represent key controlling factors for such computing platforms, namely, the limited RRAM cell programmable conductance levels, which limits the number of bits stored in one RRAM cell, the ADC noise converting analog values to digital domain and the ADC power scaling with the number of bits of its output. To address these points, in this article, we propose a framework to enhance the robustness of neural networks under variations. First, an enhanced Lipschitz constant regularization is adopted during neural network training to suppress the amplification of errors propagated through network layers. Additionally, the quantization setting of a NN model is optimized considering robustness against weight variations and total ADC power consumption. Afterward, error compensation is introduced at necessary locations determined by reinforcement learning (RL) to rescue the feature maps with remaining errors. Experimental results demonstrate that inference accuracy of neural networks can be recovered from as low as 1.69% under variations back to more than 95% of their original accuracy at the highest level of variations and reducing total ADC power consumption by 55% while the training and hardware cost are negligible.

We consider an ensemble of constant composition codes that are subsets of linear codes: while the encoder uses only the constant-composition subcode, the decoder operates as if the full linear code was used, with the motivation of simultaneously benefiting both from the probabilistic shaping of the channel input (to achieve higher rates) and from the linear structure of the code (to allow for lower complexity practical decoding). We prove that the codebook mismatch can be fully compensated by using a mismatched additive decoding metric that achieves the random coding error exponent of (non-linear) constant composition codes. As the coding rate tends to the mutual information, the optimal mismatched metric approaches the maximum a posteriori probability (MAP) metric, showing that codebook mismatch with MAP metric is capacity-achieving for the optimal input assignment.

This work shows how non-binary low-density parity-check codes over GF(2^p) can be combined with probabilistic amplitude shaping (PAS) (Böcherer, et al., 2015), which combines forward-error correction with non-uniform signaling for power-efficient communication. Ultra-sparse low-density parity-check codes over GF(64) and GF(256) gain 0.6 dB in power efficiency over state-of-the-art binary LDPC codes at a spectral efficiency of 1.5 bits per channel use and a blocklength of 576 bits. The simulation results are compared to finite length coding bounds and complemented by density evolution analysis.

The capacity-achieving input distribution of the discrete-time, additive white Gaussian noise (AWGN) channel with an amplitude constraint is discrete and seems difficult to characterize explicitly. A dual capacity expression is used to derive analytic capacity upper bounds for scalar and vector AWGN channels. The scalar bound improves on McKellips' bound and is within 0.1 bit of capacity for all signal-to-noise ratios (SNRs). The 2-D bound is within 0.15 bits of capacity provably up to 4.5 dB; numerical evidence suggests a similar gap for all SNRs. As the SNR tends to infinity, these bounds are accurate and match with a volume-based lower bound. For the 2-D complex case, an analytic lower bound is derived by using a concentric constellation and is shown to be within 1 bit of capacity.

A protograph-based low-density parity-check (LDPC) code design technique for bandwidth-efficient coded modulation with probabilistic shaping is presented. The approach jointly optimizes the LDPC code node degrees and the mapping of the coded bits to the bit-interleaved coded modulation (BICM) bit-channels. For BICM with uniform inputs and for BICM with probabilistic shaping, binary-input symmetric-output surrogate channels for the code design are used. The constructed codes for uniform inputs perform as good as the multi-edge type codes of Zhang and Kschischang (2013). For 8-ASK and 64-ASK with probabilistic shaping, codes of rates 2/3 and 5/6 with blocklength 64800 are designed, which operate within 0.63 and 0.69 dB of 1/2 log2(1 + SNR) for a target frame error rate of 10-3at spectral 1 efficiencies of 1.38 and 4.25 bits/channel use, respectively.

Finite precision approximations of discrete probability distributions are considered, applicable for distribution synthesis, e.g., probabilistic shaping. Two algorithms are presented that find the optimal M-type approximation Q of a distribution P in terms of the variational distance II Q - PII1and the informational divergence D( QIIP). Bounds on the approximation errors are derived and shown to be asymptotically tight. Several examples illustrate that the variational distance optimal approximation can be quite different from the informational divergence optimal approximation.

Distribution matching transforms independent and Bernoulli(1/2) distributed input bits into a sequence of output symbols with a desired distribution. Fixed-to-fixed length, invertible, and low complexity encoders and decoders based on constant composition and arithmetic coding are presented. The encoder achieves the maximum rate, namely, the entropy of the desired distribution, asymptotically in the blocklength. Furthermore, the normalized divergence of the encoder output and the desired distribution goes to zero in the blocklength.

A protograph-based low-density parity-check (LDPC) code design technique for bandwidth-efficient coded modulation is presented. The approach jointly optimizes the LDPC code node degrees and the mapping of the coded bits to the bit-interleaved coded modulation (BICM) bit-channels. For BICM with uniform input and for BICM with probabilistic shaping, binary-input symmetric-output surrogate channels are constructed and used for code design. The constructed codes perform as good as multi-edge type codes of Zhang and Kschischang (2013). For 64-ASK with probabilistic shaping, a blocklength 64800 code is constructed that operates within 0.69 dB of 1 over 2 log2(1 + SNR) at a spectral efficiency of 4.25 bits/channel use and a frame error rate of 10-3.

The capacity-achieving input distribution of the discrete-time additive white Gaussian noise (AWGN) channel with an amplitude constraint is discrete and seems difficult to characterize explicitly. A dual capacity expression is used to derive analytic capacity upper bounds for scalar and vector AWGN channels. The scalar bound improves on McKellips' bound and is within 0.1 bits of capacity for all signal-to-noise ratios (SNRs). The two-dimensional bound is within 0.15 bits of capacity provably up to 4.5 dB, and numerical evidence suggests a similar gap for all SNRs.

A new coded modulation scheme is proposed. At the transmitter, the concatenation of a distribution matcher and a systematic binary encoder performs probabilistic signal shaping and channel coding. At the receiver, the output of a bitwise demapper is fed to a binary decoder. No iterative demapping is performed. Rate adaption is achieved by adjusting the input distribution and the transmission power. The scheme is applied to bipolar amplitudeshift keying (ASK) constellations with equidistant signal points and it is directly applicable to two-dimensional quadrature amplitude modulation (QAM). The scheme is implemented by using the DVB-S2 low-density parity-check (LDPC) codes. At a frame error rate of 10-3, the new scheme operates within less than 1.1 dB of the AWGN capacity 1/2 log2(1 + SNR) at any spectral efficiency between 1 and 5 bits/s/Hz by using only 5 modes, i.e., 4-ASK with code rate 2/3, 8-ASK with 3/4, 16-ASK and 32-ASK with 5/6, and 64-ASK with 9/10.

A scheme is proposed that combines probabilistic signal shaping with bit-metric decoding. The transmitter generates symbols according to a distribution on the channel input alphabet. The symbols are labeled by bit strings. At the receiver, the channel output is decoded with respect to a bit-metric. An achievable rate is derived using random coding arguments. For the 8-ASK AWGN channel, numerical results show that at a spectral efficiency of 2 bits/s/Hz, the new scheme outperforms bit-interleaved coded modulation (BICM) without shaping and BICM with bit shaping (Guillén i Fàbregas and Martinez, 2010) by 0.87 dB and 0.15 dB, respectively, and is within 0.0094 dB of the coded modulation capacity. The new scheme is implemented by combining a distribution matcher with a systematic binary low-density parity-check code. The measured finite-length gains are very close to the gains predicted by the asymptotic theory.

Rooted trees with probabilities are used to analyze properties of variable length codes. A bound is derived on the difference between the entropy rates of such codes and memoryless sources. The bound is in terms of normalized informational divergence and is used to derive converses for exact random number generation, resolution coding, and distribution matching.

Fixed-to-variable length (f2v) matchers are used to reversibly transform an input sequence of independent and uniformly distributed bits into an output sequence of bits that are (approximately) independent and distributed according to a target distribution. The degree of approximation is measured by the informational divergence between the output distribution and the target distribution. An algorithm is developed that efficiently finds optimal f2v codes. It is shown that by encoding the input bits blockwise, the informational divergence per bit approaches zero as the block length approaches infinity. A relation to data compression by Tunstall coding is established.

The number of random bits required to approximate a target distribution in terms of un-normalized informational divergence is considered. It is shown that for a variable-to-variable length encoder, this number is lower bounded by the entropy of the target distribution. A fixed-to-variable length encoder is constructed using M-type quantization and Tunstall coding. It is shown that the encoder achieves in the limit an un-normalized informational divergence of zero with the number of random bits per generated symbol equal to the entropy of the target distribution. Numerical results show that the proposed encoder significantly outperforms the optimal block-to-block encoder in the finite length regime.

Consider the estimation of an unknown parameter vector in a linear measurement model. Centralized sensor selection consists in selecting a set of kssensor measurements, from a total number of m potential measurements. The performance of the corresponding selection is measured by the volume of an estimation error covariance matrix. In this work, we consider the problem of selecting these sensors in a distributed or decentralized fashion. In particular, we study the case of two leader nodes that perform naive decentralized selections. We demonstrate that this can degrade the performance severely. Therefore, two heuristics based on convex optimization methods are introduced, where we first allow one leader to make a selection, and then to share a modest amount of information about his selection with the remaining node. We will show that both heuristics clearly outperform the naive decentralized selection, and achieve a performance close to the centralized selection.

Bit-interleaved coded modulation (BICM) is a practical approach for reliable communication over the AWGN channel in the bandwidth limited regime. For a signal point constellation with 2mpoints, BICM labels the signal points with bit strings of length m and then treats these m bits separately both at the transmitter and the receiver. BICM capacity is defined as the maximum of a certain achievable rate. Maximization has to be done over the probability mass functions (pmf) of the bits. This is a non-convex optimization problem. So far, the optimal bit pmfs were determined via exhaustive search, which is of exponential complexity in m. In this work, an algorithm called bit-alternating convex concave method (Bacm) is developed. This algorithm calculates BICM capacity with a complexity that scales approximately as m3. The algorithm iteratively applies convex optimization techniques. Bacm is used to calculate BICM capacity of 4,8, 16, 32, and 64-PAM in AWGN. For PAM constellations with more than 8 points, the presented values are the first results known in the literature.

Many communication channels with discrete input have non-uniform capacity achieving probability mass functions (PMF). By parsing a stream of independent and equiprobable bits according to a full prefix-free code, a modulator can generate dyadic PMFs at the channel input. In this work, we show that for discrete memoryless channels and for memoryless discrete noiseless channels, searching for good dyadic input PMFs is equivalent to minimizing the Kullback-Leibler distance between a dyadic PMF and a weighted version of the capacity achieving PMF. We define a new algorithm called Geometric Huffman Coding (GHC) and prove that GHC finds the optimal dyadic PMF in O(m log m) steps where m is the number of input symbols of the considered channel. Furthermore, we prove that by generating dyadic PMFs of blocks of consecutive input symbols, GHC achieves capacity when the block length goes to infinity.

The capacity achieving probability mass function (PMF) of a finite signal constellation with an average power constraint is in most cases non-uniform. A common approach to generate non-uniform input PMFs is Huffman shaping, which consists of first approximating the capacity achieving PMF by a sampled Gaussian density and then to calculate the Huffman code of the sampled Gaussian density. The Huffman code is then used as a prefix-free modulation code. This approach showed good results in practice, can however lead to a significant gap to capacity. In this work, a method is proposed that efficiently constructs optimal prefix-free modulation codes for any finite signal constellation with average power constraint in additive noise. The proposed codes operate as close to capacity as desired. The major part of this work elaborates an analytical proof of this property. The proposed method is applied to 64-QAM in AWGN and numeric results are given, which show that, opposed to Huffman shaping, by using the proposed method, it is possible to operate very close to capacity over the whole range of parameters.

Unequal transition probabilities between input and output symbols, input power constraints, or input symbols of unequal durations can lead to non-uniform capacity achieving input distributions for communication channels. Using uniform input distributions reduces the achievable rate, which is called the shaping gap. Gallager's idea for reliable communication with zero shaping gap is to do encoding, matching, and jointly decoding and dematching. In this work, a scheme is proposed that consists in matching, encoding, decoding, and dematching. Only matching is channel specific whereas coding is not. Thus off-the-shelf LDPC codes can be applied. Analytical formulas for shaping and coding gap of the proposed scheme are derived and it is shown that the shaping gap can be made zero. Numerical results show that the proposed scheme allows to operate off-the-shelf LDPC codes with zero shaping gap and a coding gap that is unchanged compared to uniform transmission.

This paper addresses the problem of resource allocation for orthogonal frequency division multiple access (OFDMA) downlink that aims at minimizing the total transmission power under data transmission constraints. To accelerate this multiuser resource allocation with small performance loss, an efficient technique is introduced that provides the power variation of single-user water-filling when the subcarrier assignment is changed. Based on this technique, an intelligent resource allocation method for OFDMA downlink is designed. First, a good starting point is determined by estimating the cardinality of the set of subcarriers that will be assigned to each user. Second, the convergence of the resource allocation is accelerated by reassigning those subcarriers first, which have the greatest channel-to-noise ratio (CNR) variation over the users. Compared to previous works, simulations show that the presented method provides an improved balance between performance and computational complexity.