The main result of the paper is a coding theorem for time-discrete sources with a fidelity criterion on average distortion. This result applies to a subclass of the ergodic sources (stationary sources having a strong mixing property) and to the cases in which source samples are continuously distributed (analog) quantities or discrete symbols. An intuitive definition of the information content of data samples for this class of sources is then formally established and is shown to be exactly the same as Shannon's definition.

Fundamental limitations on the performance of any type of communication system transmitting data from an analog source may be obtained by application of a theorem in Shannon's original paper on information theory. In order to transmit the output of an analog source to a sink with mean-squared error e or less, it is necessary to transmit information from source to sink at rates greater than some minimal rateR(\epsilon)bits per second. If a channel of capacityCbits per second is used, then in order for any communication system to be able to achieve\epsilonmean-squared error or less with this channel it is necessary thatR(\epsilon) \leq C. This inequality can be used to determine the minimum mean-squared error obtainable for a given source-channel pair without discussing any particular communication system. As an example, the minimum mean-squared error is calculated for cases in which the analog source and additive channel noise are stationary, Gaussian processes. The performance of amplitude and angle modulation systems is compared to the theoretically ideal performance obtainable in some of these cases.