Two conservation laws for mutual information in terms of directed informations between two synchronized sequences of random variables are derived, the first for the case of no conditioning and the second for the case of causal conditioning on a third synchronized sequence. As a byproduct of the derivation of the first conservation law, the directed information specifying the feedback flowing from the second sequence to the first sequence is identified, which leads to a simple proof that a previously known sufficient condition for equality of mutual and directed information is also a necessary condition

This paper describes an iterative method of decoding (for convolutional codes) that obtains soft-estimate decoding performances in-between BP decoding and optimal MAP decoding. The idea is to replace an original even-parity function (which is used in BP decoding) with a "reduced" core function and a corresponding multistate trellis. The result is a soft-decoding performance that can be better than the performance for recursive BP decoding. The decoding method does not achieve the optimal MAP performance due to dependencies within the recursive algorithm. The benefit is that the trellis may have far fewer states than the full 2m-state trellis which is required for MAP decoding

We discuss the design of new low-complexity turbo-like codes based on a multiple parallel concatenation of 4-state and 2-state constituent codes. The new code designs take advantage of the big-numerator/little-denominator principle along with specially designed interleavers to outperform previously designed turbo codes over the entire range of signal-to-noise ratios. Puncturing at the encoder is used to produce low-complexity codes with excellent performance at code rates of 1/2 and 1/3. The multiple parallel concatenation and puncturing results in turbo-like encoders which are either partially systematic or completely nonsystematic. Comparisons with the proposed 8-state turbo coding standard and with other low-complexity alternative code designs are included.

Turbo codes with long frame lengths are usually constructed using a randomly chosen interleaver. Statistically, this guarantees excellent bit-error rate (BER) performance but also generates a certain number of low weight codewords, resulting in the appearance of an error floor in the BER curve. Several methods, including using an outer code, have been proposed to improve the error floor region of the BER curve. We study the effect of an outer BCH code on the frame-error rate (FER) of turbo codes. We show that additional coding gain is possible not only in the error floor region but also in the waterfall region. Also, the outer code improves the iterative APP decoder by providing a stopping criterion and alleviating convergence problems. With this method, we obtain codes whose performance is within 0.6 dB of the sphere packing bound at an FER of 10/sup -6/.