This paper investigates a stochastic decision problem with a stationary memoryless random sequence of states, where the maximum a posteriori decision is employed within the states. We derive novel upper bounds on the decision error probability depending on the sequence length. It is shown that, when a sequence is generated subject to an a posteriori distribution, the ratio of the error probability divided by the error probability of the maximum a posteriori decision approaches one as the sequence length increases.

A construction of a channel code by using a source code with decoder side information is introduced. The encoder and decoder pair of any source code can be used for the construction. Constrained-random-number generators, which generate random numbers satisfying a condition specified by a function and its value, are used to construct stochastic encoders and decoders. The result suggests that we can divide the channel coding problem into the problems of channel encoding and source decoding with side information.

A general multi-terminal source code and a general multi-terminal channel code are presented. Constrained-random-number generators with sparse matrices, which are building blocks for the code construction, are used in the construction of both encoders and decoders. Achievable regions for source coding and channel coding are derived in terms of entropy functions, where the capacity region for channel coding provides an alternative to the region of [Somekh-Baruch and Verdú, ISIT2006].

This paper investigates the error probability of a stochastic decision and the way in which it differs from the error probability of an optimal decision, i.e., the maximum a posteriori decision. This paper calls attention to the fact that the error probability of a stochastic decision with the a posteriori distribution is at most twice the error probability of the maximum a posteriori decision. It is shown that, by generating an independent identically distributed random sequence subject to the a posteriori distribution and making a decision that maximizes the a posteriori probability over the sequence, the error probability approaches exponentially the error probability of the maximum a posteriori decision as the sequence length increases. Using these ideas as a basis, we can construct stochastic decoders for source/channel codes.

There is a flaw in the statement of Lemma 5 in the above titled paper (ibid., vol. 56, no. 5, pp. 2143-2167, May 2010), which is used in the proof of Ths. 4, 6, and 7 in the same paper. Revisions are presented here.

An LDPC matrix is used as a local encoding kernel at each link to construct a universal code to address network coding problems. It is also shown that at each terminal node the global encoding kernel that constructs a decoder becomes an LDPC matrix. This provides the perspective that decoding complexity can be reduced to a linear order of a block length by using an efficient decoding algorithm such as the sum-product algorithm.

We investigate the problem of (independent) uniform random number generation and secret key agreement. A generator for the (independent) uniform random numbers is constructed by using a sparse matrix and it is applied to a secret key agreement protocol with strong secrecy. It is proved that the rate of the proposed codes can achieve the fundamental limits.

The aim of this paper is to prove coding theorems for the wiretap channel and the secret key agreement based on the the notion of a hash property for an ensemble of functions. These theorems imply that codes using sparse matrices can achieve the optimal rate. Furthermore, fixed-rate universal coding theorems for a wiretap channel and a secret key agreement are also proved.

There are flaws in the proof of Ths. 1 and 3 in the above titled paper (ibid., vol 56, no. 6, pp. 2688-2698, Jun. 2010). Corrections are provided here.

A strongly secure wiretap channel code is proposed. The construction is based on the strong hash property introduced in Proc. ISIT2010, pp. 575-579. Since an ensemble of sparse matrices satisfies the conditions for the strong hash property, the rate of the proposed code using sparse matrices can achieve the secrecy capacity.

The aim of this paper is to introduce the construction of codes for multiple access channels based on the the notion of the strong hash property introduced in Proc. ISIT2010, pp. 575-579. Since an ensemble of sparse matrices has a strong hash property, we can use sparse matrices for code constructions.

The aim of this paper is to construct a code for broadcast channel (independent messages and no common message) based on the the notion of a stronger version of the hash property for an ensemble of functions. Since an ensemble of sparse matrices has a strong hash property, codes using sparse matrices can achieve an inner bound of capacity region.

The aim of this paper is to prove the achievability of rate regions for several coding problems by using sparse matrices (with logarithmic column degree) and maximum-likelihood (ML) coding. These problems are the Gel'fand-Pinsker problem, the Wyner-Ziv problem, and the one-helps-one problem (source coding with partial side information at the decoder). To this end, the notion of a hash property for an ensemble of functions is introduced and it is proved that an ensemble of q-ary sparse matrices satisfies the hash property. Based on this property, it is proved that the rate of codes using sparse matrices and ML coding can achieve the optimal rate.

The aim of this paper is to prove the fixed-rate universal coding theorems by using the notion of the hash property. These theorems are the fixed-rate lossless universal source coding theorem and the fixed-rate universal channel coding theorem. Since an ensemble of sparse matrices (with logarithmic column degree) satisfies the hash property requirement, it is proved that we can construct universal codes by using sparse matrices.

In the above titled paper (ibid., vol. 56, no. 5, pp. 2143-2167, May 10, there is a gap in the proof of Theorem 4. The corrections are provided here.

The aim of this paper is to prove the achievability of the general (asymmetric) channel coding problem based on the hash property. Since an ensemble of q-ary sparse matrices (the maximum column weight grows logarithmically in the block length) satisfies the hash property, it is proved that the rate of codes using sparse matrices can achieve the channel capacity.

A channel code is constructed using sparse matrices for stationary memoryless channels that do not necessarily have a symmetric property like a binary symmetric channel. It is also shown that the constructed code has the following remarkable properties: 1) Joint source-channel coding: Combining with lossy source code, which is also constructed by sparse matrices, a simpler joint source-channel code can be constructed than that constructed by the ordinary block code. 2) Universal coding: The constructed channel code has a universal property under a specified condition.

The aim of this paper is to prove the achievability of the Wyner-Ziv source coding problem by using sparse matrices and maximal-likelihood (ML) coding. To this end, the notion of a hash property for an ensemble of functions is introduced. For example, an ensemble of q-ary sparse matrices satisfies the hash property. Based on this property, it is proved that the rate of codes using sparse matrices and maximal-likelihood (ML) coding can achieve the optimal rate.

Research into applying LDPC code theory, which is used for channel coding, to source coding has received a lot of attention in several research fields such as Distributed Source Coding. In this paper a source coding problem with a fidelity criterion is considered. Matsunaga et al. [6] constructed a lossy code under the conditions of a binary alphabet, a uniform distribution, and a Hamming measure of fidelity criterion. We extend their results and construct a lossy code under the extended conditions of a binary alphabet, a distribution that is not necessarily uniform, and a fidelity measure that is bounded and additive and show that the code can achieve the optimal rate, rate-distortion function.

For cases other than the binary alphabet, we construct an LDPC code with the fidelity criterion for uniformly distributed sources on Zq, which asymptotically achieves the rate-distortion limit. For the existence of the LDPC code, we derive some conditions for the distortion criterion and for the parameters of the parity check matrix

In the above-titled paper (ibid., vol.2, p.471-475, July 1991), the authors claim that neural network classifiers duplicate the decision rule created by the empirical Bayes rule. The commenter states that this statement is, in fact, not generally true and points out an error in the proof. The commenter also shows that the related true statement about the relation between neural and Bayes classifiers is of no practical use. In reply, the authors show that the example that the commenter uses to disprove the equation is misleading. They refute the commenter's second point as well.

It is demonstrated both theoretically and experimentally that, under appropriate assumptions, a neural network pattern classifier implemented with a supervised learning algorithm generates the empirical Bayes rule that is optimal against the empirical distribution of the training sample. It is also shown that, for a sufficiently large sample size, asymptotic equivalence of the network-generated rule to the theoretical Bayes optimal rule against the true distribution governing the occurrence of data follows immediately from the law of large numbers. It is proposed that a Bayes statistical decision approach leads naturally to a probabilistic definition of the valid generalization which a neural network can be expected to generate from a finite training sample.

Generalized mean-squared error (GMSE) objective functions are proposed that can be used in neural networks to yield a Bayes optimal solution to a statistical decision problem characterized by a generic loss function.