Source coding theorems for general sources are presented. For a source /spl mu/, which is assumed to be a probability measure on all strings of an infinite-length sequence with a finite alphabet, the notion of almost-sure sup entropy rate is defined; it is an extension of the Shannon entropy rate. When both an encoder and a decoder know that a sequence is generated by /spl mu/, the following two theorems can be proved: (1) in the almost-sure sense, there is no variable-rate source coding scheme whose coding rate is less than the almost-sure sup entropy rate of /spl mu/, and (2) in the almost-sure sense, there exists a variable-rate source coding scheme whose coding rate achieves the almost-sure sup entropy rate of /spl mu/.

This article deals with the problem of simulating a discrete memoryless channel and proposes two algorithms for channel simulation by using the interval algorithm. The first algorithm provides exact channel simulation and the number of fair random bits per input sample approaches the conditional resolvability of the channel with probability one. The second algorithm provides approximate channel simulation and the approximation error measured by the variational distance vanishes exponentially as the block length tends to infinity, when the number of fair random bits per input sample is above the conditional resolvability. Further, some asymptotic properties of these algorithms as well as the original interval algorithm for random number generation are clarified.

We consider the Bayesian parameter estimation problem where the value of a finitary parameter X should be decided on the basis of i.i.d. sample Y/sup n/ of size n. In this context, the amount of missing information on X after observing Y/sup n/ may be evaluated by the posterior entropy, which is often called the equivocation or the conditional entropy, of X given Y/sup n/, while it is well known that the minimum possible probability of error in estimating X is achieved by the maximum a posteriori probability (MAP) estimator. In this work, the focus is on the asymptotic relation between the posterior entropy and the MAP error probability as the sample size n becomes sufficiently large. It is shown that if the sample size n is large enough, the posterior entropy as well as the MAP error probability decay with n to zero at the identical exponential rate, and that the maximum achievable exponent for this decay is determined by the minimum Chernoff information over all the possible pairs of distinct parameter values.

Simple statistical hypothesis testing is investigated by making use of the divergence geometric method. The asymptotic behavior of the minimum value of the error probability of the second kind under the constraint that the error probability of the first kind is bounded above by exp(-rn) is looked for, where r is a given positive number. If r is greater than the divergence of the two probability measures, the so-called converse theorem holds. It is shown that the condition under which the converse theorem holds can be divided into two separate cases by analyzing the geodesic connecting the two probability measures, and, as a result, an explanation is given for the Han-Kobayashi linear function f/sub T/(X).>

Hypothesis testing for two Markov chains is considered. Under the constraint that the error probability of the first kind is less than or equal to exp(-rn), the error probability of the second kind is minimized. The geodesic that connects the two Markov chains is defined. By analyzing the geodesic, the power exponents are calculated and then represented in terms of Kullback-Leibler divergence.>

In the above-titled paper (ibid., vol.2, p.471-475, July 1991), the authors claim that neural network classifiers duplicate the decision rule created by the empirical Bayes rule. The commenter states that this statement is, in fact, not generally true and points out an error in the proof. The commenter also shows that the related true statement about the relation between neural and Bayes classifiers is of no practical use. In reply, the authors show that the example that the commenter uses to disprove the equation is misleading. They refute the commenter's second point as well.

A basic mathematical function that conjoins the two key conceptions of mutual information and Bayes risk is defined. Based on that function, some asymptotic theorems that verify an important implication of mutual information in the context of practical Bayesian decisionmaking are proven.>

It is demonstrated both theoretically and experimentally that, under appropriate assumptions, a neural network pattern classifier implemented with a supervised learning algorithm generates the empirical Bayes rule that is optimal against the empirical distribution of the training sample. It is also shown that, for a sufficiently large sample size, asymptotic equivalence of the network-generated rule to the theoretical Bayes optimal rule against the true distribution governing the occurrence of data follows immediately from the law of large numbers. It is proposed that a Bayes statistical decision approach leads naturally to a probabilistic definition of the valid generalization which a neural network can be expected to generate from a finite training sample.

Generalized mean-squared error (GMSE) objective functions are proposed that can be used in neural networks to yield a Bayes optimal solution to a statistical decision problem characterized by a generic loss function.

The effect of channel losses on the capacity of noiseless optical communication systems utilizing the number state is studied. Based on these findings, the photon efficiency of these systems is evaluated in the presence of losses. Then, by comparing the photon efficiency of the number-state pulse-position modulation (PPM) and the coherent-state PPM under certain realistic conditions, it is found that the former is always superior to the latter from the perspective of photon efficiency regardless of losses.>

Maximizing the mutual information under a linear input constraint is considered from a geometric point of view. Assuming a suitable regularity condition on the channel matrix, it is found that the probability distribution (PD) equidistant from the row PDs of the channel matrix plays an important role, and the maximum is achieved by the projection of that PD onto the set of PDs satisfying the constraint. The PD attaining the capacity-constraint function is obtained by using Lagrange's method of indeterminate coefficients at most (m-2) times.>

A novel geometrical characterization of the capacity of a discrete memoryless channel is proposed. It is based on Csiszar's theorem, which represents the capacity using the Kullback-Leibler discrimination information. A new geometrical capacity computing method is obtained.>

Digital data services have been offered by NTT since 1978. The largest part of the entire digital data network cost is attributable to subscriber loop transmission that includes digital service units (DSU's) and office channel units (OCU's). Therefore, it is important to reduce DSU and OCU costs for economizing on digital data services. In order to achieve this purpose, new LSI DSU and OCU have been developed. The LSI is used for a digital signal processing unit, which occupies most of the equipment cost and size. The LSI can be applied to all bearer rates from 3.2 to 64 kbits/s, for NTT digital data services (leased circuit service, circuit switched service and packet switched service). It is highly significant that adoption of the LSI in DSU and OCU makes the following estimations feasible: 1) About 30 to 40 percent Cost reduction 2) About 25 percent size reduction 3) About 30 to 50 percent failure rate reduction 4) About 50 to 70 percent power consumption reduction. Summary The most expensive part of the network that carries NTT Digital Data Services is the subscriber loop transmission system, which includes DSU and OCU. This paper describes the design philosophy for introducing LSI technology in the DSU and OCU for cost reduction. Then the function, construction and evaluation are presented for the newly developed LSI DSU and OCU, based on the above design concept.

An interframe coding system is described which is capable of transmitting both 4-MHz videotelephone signals and NTSC color TV signals at an average rate of 6.312 Mbits/s. The coding system takes as much advantage as possible of in-frame correlation, in addition to interframe correlation, by coding pel-to-pel difference of interframe difference (combinational difference). Variable-length word coding is effectively used to reduce the number of bits to be transmitted. In order to prevent buffer overflow, luminance and spatial resolutions are controlled. To control luminance resolution, interframe difference is multiplied by a coefficient\alpha \leq 1, so that value approaches zero when violent motion causes the buffer to fill. Though multiplication with small α results in temporal resolution reduction in moving areas, quality degradation is not too marked. Spatial resolution control is achieved by using subsample and subline coding modes.

As a powerful tool for economizing on the digital transmission of videotelephone signals, interframe coding techniques have drawn increasing attention, and various interframe coding schemes have been proposed recently. However, many problems remain to be studied in establishing technical feasibility for these techniques and schemes. This concise paper reports on field trial in which a 1.544-Mbits/s digital transmission of videotelephone signals was conducted over Nippon Telegraph and Telephone Public Corporation's (NTT's) PCM100M digital repeatered line between Tokyo and Yokohama, Japan. The experimental interframe codec is based on the well-understood conditional picture element replenishment algorithm. The object of the field trial was to demonstrate, not the validity of the algorithm itself, but the technical feasibility of the interframe coding system as an entity. Experimental results are satisfactory enough to assure us of the system's practicality in the near future.