The basic properties of Renyi's entropy are reviewed, and its concavity properties are characterized. New bounds (referred to asI_{\alpha}bounds) on the probability of error are derived from Renyi's entropy and are compared with known bounds. It is proved that for the two-class case, theI_{2}bound is sharper than many of the previously known bounds. The difference between theI_{2}bound and the real value of the probability of error is at most 0.09.

The general problem of estimating the a posteriori probabilities of the states and transitions of a Markov source observed through a discrete memoryless channel is considered. The decoding of linear block and convolutional codes to minimize symbol error probability is shown to be a special case of this problem. An optimal decoding algorithm is derived.

Relationships between the probability of error, the equivocation, and the Chernoff bound are examined for the two-hypothesis decision problem. The effect of rejections on these bounds is derived. Finally, the results are extended to the case of any finite number of hypotheses.

In many pattern-recognition problems there exist dependencies among the patterns to be recognized. In the past, these dependencies have not been introduced into the mathematical model when designing an optimal pattern-recognition system. In this paper the optimal decision rule is derived under the assumption of Markov dependence among the patterns to be recognized. Subsequently, this decision rule is applied to character-recognition problems. The main idea is to balance appropriately the information which is obtained from contextual considerations and the information from measurements on the character being recognized and thus arrive at a decision using both. Bayes' decision in Markov chains is presented and this mode of decision is adapted to character recognition. A look-ahead mode of decision is presented. The problem of estimation of transition probabilities is discussed. The experimental system is described and results of experiments on English legal text and names are presented.