Heredity is relevant to information theory as a communication process. The conservation of genomes over intervals at the geological timescale and the existence of mutations at shorter intervals can be conciliated assuming that genomes possess intrinsic error-correction codes. The better conservation of old parts of genomes leads to assume that these codes are organized as a nested system set up during geological times, which protects a genomic message the better, the older it is. These hypotheses imply that: genomes are redundant, discrete species exist with a hierarchical taxonomy, successive generations are needed, evolution is contingent and saltationist; it trends towards increasing complexity. These consequences match features of the actual living world but their experimental confirmation needs a still lacking collaboration of biologists and information-theorists. It is suggested that genomic error-correcting codes could consist of ¿soft codes¿ where mutual dependence of symbols results from physical-chemical and linguistic constraints, not only mathematical equalities. The constraints incurred by DNA molecules moreover result in a nested structure. Guesses about genomic error-correcting codes are made.

Information theory--a field at the intersection of applied mathematics and electrical engineering--was primarily developed for the purpose of addressing problems arising in data storage and data transmission over (noisy) communication media. Consequently, information theory provides the formal basis for much of today’s storage and communication infrastructure.

Replication decoding is intended to exploit the redundancy of a sequence of symbols so as to perform error correction. Mutual dependence, as implied by redundancy, enables computing several replicas of each symbol. Replication decoding optimally takes account of all such replicas. Its application to linear codes is reviewed. Replication decoding can also be used for 'soft codes', a probabilistic extension of redundant codes. As an example, its possible role in the mechanisms which maintain the genetic message borne by the DNA molecule is considered.

For understanding turbo codes, we propose to locate them at the intersection of three main topics: the random-like criterion for designing codes, the idea of extending the decoding role to reassess probabilities, and that of combining several codes by product or concatenation. Concerning the idea of designing random-like (RL) codes, we distinguish strongly and weakly random-like codes depending on how the closeness of their weight distribution to that obtained in the average by random coding is measured. Using, e.g., the cross entropy as a closeness measure results in weakly RL codes. Although their word-error rate is bad, their bit-error rate (BER) remains low up to the vicinity of the channel capacity. We show that pseudorandom recursive convolutional codes belong to this family. Obtaining reasonably good performance with a single code of this type involves high complexity, and its specific decoding is difficult. However, using these codes as components in the turbo-code scheme is a simple means for improving the low-weight tail of the distribution and to adjust the BER to any specification. In order to increase the encoder memory without inordinate complexity, it is suggested to use iterated nonexhaustive replication decoding.

Any symbol in a redundant code can be recovered when it belongs to certain erasure patterns. Several alternative expressions of a given symbol, to be referred to as its replicas, can therefore be computed in terms of other ones. Decoding is interpreted as decoding upon a received symbol, given itself and a number of such replicas, expressed in terms of other received symbols. For linearq-ary (n,k)block codes, soft-decision demodulation and memoryless channels, the maximum-likelihood decision rule on a given symbol is formulated in terms ofr \leqn - klinearly independent replicas from the parity-check equations. All replicas deriving from therselected replicas by linear combination are actually taken into account in this decision rule. Its implementation can be direct; use transformations or a sequential circuit implementing a trellis representation of the parity-check matrix. Ifr = n - k, decoding is optimum, in the sense of symbol-by-symbol maximum-likelihoed. Simplification results in the transformed and sequential implementations whenr < n - k. If the selected replicas are disjoint, generalized (q-ary, weighted) threshold decoding results. The decoding process can easily be modffied in order to provide word-by-word maximum-likelihood decoding. Convolutional codes are briefly considered. Two specific problems are discussed: the use of previous decisions, which leads to a weighted generalization of feedback decoding, and the extension of replication decoding to nonsystematic codes.

This study is devoted to approximately determine the reception threshold position in large lncex frequency modulation, as reached by the use of a hypothetical (yet unknown) optimum demodulator. First, an interpretation of threshold phenomenon is given, in the case of a continuous modulation system with modulation gain (i.e., which trades minimum necessary power against bandwidth). After SHANNON, demodulation is then represented as a non-topological mapping in high-dimensional signal spaces. For a continuous modulation system, however, it is locally topological (in a neighbourhood of the signal point). One may thus share the noise after demodulation into ,neighbourhood and noise, whether the signal plus noise point remains or not in the locally topological neighbourhood of the signal point. The former is predominant above threshold, the latter below; threshold is defined by the equality of their powers. In the FM case, neighbourhood noise is already known, according to the conventional large signal theory. Error noise may be approximately computed after replacing the actual continuous modulating signal by a discontinuous one, in such a way that: a) the performance, as far as error noise is concerned, remains essentially unchanged; b) but the optimum demodulator is then explicitely known (optimum in the sense of maximum likelihood, in the presence of stationary white gaussian noise), and allows thus for the explicit computation of the extreme threshold position. For this purpose, the modulating signal is a) quantized (this is justified by the discontinuous nature of the error noise); b) sampled at the NYQUIST rate, each of the samples being held till the following occurs (this is justified with the help of basic principles of statistical communication theory). The modulated signal thus obtained with a L -levels quantization is (for large index, and a somewhat rough choice of L ) a (quasi-)orthogonal L -ary signal, for which explicit calculation of error noise is easy. Its results show that a large improvement beyond the threshold of limiter-discriminator type demodulators is possible, though not as large as sometimes written about frequencyfeedback demodulators. A physical interpretation of these results is them attempted, by the use of the concept of short term spectrum. Finally, a critical review of the frequency feedback type demodulators is given, and a novel type of demodulator is proposed, which is believed to more closely approach the theoretical limit than does the frequencyfeedback type.

This study is devoted to approximately determine the reception threshold position in large lncex frequency modulation, as reached by the use of a hypothetical (yet unknown) optimum demodulator. First, an interpretation of threshold phenomenon is given, in the case of a continuous modulation system with modulation gain (i.e., which trades minimum necessary power against bandwidth). After SHANNON, demodulation is then represented as a non-topological mapping in high-dimensional signal spaces. For a continuous modulation system, however, it is locally topological (in a neighbourhood of the signal point). One may thus share the noise after demodulation into ,"neighbourhood" and "error" noise, whether the signal plus noise point remains or not in the locally topological neighbourhood of the signal point. The former is predominant above threshold, the latter below; threshold is defined by the equality of their powers. In the FM case, neighbourhood noise is already known, according to the conventional large signal theory. Error noise may be approximately computed after replacing the actual continuous modulating signal by a discontinuous one, in such a way that: a) the performance, as far as error noise is concerned, remains essentially unchanged; b) but the optimum demodulator is then explicitely known (optimum in the sense of maximum likelihood, in the presence of stationary white gaussian noise), and allows thus for the explicit computation of the extreme threshold position. For this purpose, the modulating signal is a) quantized (this is justified by the discontinuous nature of the error noise); b) sampled at the NYQUIST rate, each of the samples being held till the following occurs (this is justified with the help of basic principles of statistical communication theory). The modulated signal thus obtained with aL-levels quantization is (for large index, and a somewhat rough choice ofL) a (quasi-)orthogonalL-ary signal, for which explicit calculation of error noise is easy. Its results show that a large improvement beyond the threshold of limiter-discriminator type demodulators is possible, though not as large as sometimes written about frequencyfeedback demodulators. A physical interpretation of these results is them attempted, by the use of the concept of "short term" spectrum. Finally, a critical review of the frequency feedback type demodulators is given, and a novel type of demodulator is proposed, which is believed to more closely approach the theoretical limit than does the frequencyfeedback type.