This paper investigates distributed source coding, multiple description coding, and source coding with side information at decoders. The equivalence between the multiple-decoder extension of distributed source coding with side information at decoders and the multiple-source extension of multiple description coding with side information at decoders is clarified. Their multi-letter rate-distortion region is characterized by entropy functions for arbitrary general correlated sources.

This paper investigates the general distributed loss-less/lossy source coding formulated by Jana and Blahut. Their multi-letter rate-distortion region, an alternative to the region derived by Yang and Qin, is characterized by entropy functions for arbitrary general correlated sources. Achievability is shown by constructing a code based on constrained-random number generators.

This paper provides a simple proof of the multi-letter converse theorem for distributed lossless source coding (Slepian-Wolf source coding and Wyner-Ahlswede-Körner source coding), where the number of reproduced sources is arbitrary. It is shown only by using basic inequalities of limit superior/inferior in probability, the non-negativity of the divergence, and the single-source version of the Fano inequality, where the epsilon-delta arguments are hidden in the above inequalities.

Channel codes for relayless networks with the general message access structure is introduced. It is shown that the multi-letter characterized capacity region of this network is achievable with this code. The capacity region is characterized in terms of entropy functions and provides an alternative to the regions introduced by [Somekh-Baruch and Verdú, ISIT2006][Muramatsu and Miyake, ISITA2018].

This paper introduces techniques to construct binary polar source/channel codes based on the bit error probability of successive-cancellation decoding. Techniques to compute the bit error probability based on the symmetric parameterization of joint sources are introduced. These techniques can be applied to the construction of polar codes and the computation of lower and upper bounds of the block decoding error probability.

This paper introduces algorithms for the successive-cancellation decoding and the successive-cancellation list decoding of binary polar source/channel codes. By using the symmetric parametrization of conditional probability, we reduce both space and time complexity compared to the original algorithm introduced by Tal and Vardy.

This paper investigates a stochastic decision problem with a stationary memoryless random sequence of states, where the maximum a posteriori decision is employed within the states. We derive novel upper bounds on the decision error probability depending on the sequence length. It is shown that, when a sequence is generated subject to an a posteriori distribution, the ratio of the error probability divided by the error probability of the maximum a posteriori decision approaches one as the sequence length increases.

This paper investigates the error probability of several decoding methods for a source code with decoder side information, where the decoding methods are: 1) symbol-wise maximum a posteriori decoding, 2) successive-cancellation decoding, and 3) stochastic successive-cancellation decoding. The proof of the effectiveness of a decoding method is reduced to that for an arbitrary decoding method, where `effective' means that the error probability goes to zero as the block length goes to infinity. Furthermore, we revisit the polar source code showing that stochastic successive-cancellation decoding, as well as successive-cancellation decoding, is effective for this code.

A construction of a channel code by using a source code with decoder side information is introduced. The encoder and decoder pair of any source code can be used for the construction. Constrained-random-number generators, which generate random numbers satisfying a condition specified by a function and its value, are used to construct stochastic encoders and decoders. The result suggests that we can divide the channel coding problem into the problems of channel encoding and source decoding with side information.

A general multi-terminal source code and a general multi-terminal channel code are presented. Constrained-random-number generators with sparse matrices, which are building blocks for the code construction, are used in the construction of both encoders and decoders. Achievable regions for source coding and channel coding are derived in terms of entropy functions, where the capacity region for channel coding provides an alternative to the region of [Somekh-Baruch and Verdú, ISIT2006].

This paper presents a dynamical model of overconfidence phenomena due to a confirmation bias. The model assumes an informational environment where a person updates one's belief about a state of the world through signals which may be misread and ignored due to a confirmation bias. A general type of confirmation bias, called ZE-type, is introduced and formalized using a model of noisy channels between signals and perceptions. It contains two subtypes of confirmation biases, called Z-and E-types, as its special cases. Persons suffering from Z-type receive a signal and perceive it incorrectly as a different signal with a certain probability, while those suffering from E-type fail to receive a signal with a certain probability. This paper derives a key factor, called a filtering coefficient, which plays a central role in analyzing the overconfidence phenomena. The analytical results suggest that the substantial part of the overconfidence phenomena due to the confirmation bias is regarded as a switched system with multiple linear time-invariant subsystems.

This paper investigates the error probability of a stochastic decision and the way in which it differs from the error probability of an optimal decision, i.e., the maximum a posteriori decision. This paper calls attention to the fact that the error probability of a stochastic decision with the a posteriori distribution is at most twice the error probability of the maximum a posteriori decision. It is shown that, by generating an independent identically distributed random sequence subject to the a posteriori distribution and making a decision that maximizes the a posteriori probability over the sequence, the error probability approaches exponentially the error probability of the maximum a posteriori decision as the sequence length increases. Using these ideas as a basis, we can construct stochastic decoders for source/channel codes.

This paper presents a preliminary analysis of an active Bayesian observer that communicates with humans to correct the overconfidence effects due to their confirmatory biases. Two types of confirmatory biases, called Z- and E-types, are introduced and formalized using a model of noisy channels between signals and perceptions. Persons suffering from Z-type confirmatory bias receive a signal and perceive it incorrectly as a different signal with a certain probability, while those suffering from E-type confirmatory bias fail to receive a signal with a certain probability. A dynamic Bayesian network model is developed to analyze the effects of Z- and E-types in a unified way. The analytical model enables us to give a theoretical insight into some basic properties of the active Bayesian observers.

Summary form only given. A variable-length lossy compression algorithms for a stationary memory less source with a continuous alphabet are introduced with a rate-distortion pair close to the rate-distortion function.

This paper reviews an approach to secret-key distribution based on the bounded observability (BO) model. First, the information-theoretic framework of secret-key agreement from a correlated random source is reviewed. Next, the BO model is introduced. In the context of this model, the BO condition is presented as a necessary and sufficient condition for the possibility of secret-key distribution. This condition describes limits on the information obtained by observation of a random object, and models the practical difficulty of completely observing random physical phenomena. Finally, an implementation of secret-key distribution based on BO in an optical fiber system is described.

A variable-length lossy source code is introduced with a rate-distortion pair close to the rate-distortion function. Random numbers that satisfy a condition specified by a function and its value are used to construct a stochastic encoder. The proof of the theorem is based on the balanced-coloring property of an ensemble of functions. Since an ensemble of systematic sparse matrices has this property, we can construct a tractable code for a memoryless source. Some algorithms for implementing the code are introduced and compared by simulation.

The coding problem for a wiretap channel with a noncausal state is investigated, where the problem includes the coding problem for a channel with a noncausal state, which is known as the Gel'fand-Pinsker problem, and the coding problem for a wiretap channel introduced by Wyner. The secrecy capacity for this channel is derived, where an optimal code is constructed based on the hash property and a constrained-random-number generator. Since an ensemble of sparse matrices has a hash property, the rate of the proposed code using a sparse matrix can achieve the secrecy capacity.

A variable-length lossy source code is introduced with a rate-distortion pair close to the rate-distortion function. The proof of the theorem is based on the balanced-coloring property of an ensemble of functions. Since an ensemble of systematic sparse matrices has this property, we can construct a practical code for a memoryless source by using the sum-product algorithm.

Stochastic encoders for channel coding and lossy source coding are introduced with a rate close to the fundamental limits, where the only restriction is that the channel input alphabet and the reproduction alphabet of the lossy source code are finite. Random numbers, which satisfy a condition specified by a function and its value, are used to construct stochastic encoders. The proof of the theorems is based on the hash property of an ensemble of functions, where the results are extended to general channels/sources and alternative formulas are introduced for channel capacity and the rate-distortion region. Since an ensemble of sparse matrices has a hash property, we can construct a code by using sparse matrices.

A stochastic encoder for lossy source coding is introduced with a rate-distortion pair close to the boundary of the rate-distortion region, where the only restriction is that the reproduction alphabet is finite. Random numbers, which satisfy a condition specified by a function and its value, are used to construct the stochastic encoder. The proof of the theorem is based on the hash property of an ensemble of functions, where the results are extended to a general channel by deriving an alternative formula for the rate-distortion region. Since an ensemble of sparse matrices has a hash property, we can construct a code by using sparse matrices.

A stochastic encoder for channel coding is introduced with a rate close to the channel capacity, where the only restriction is that the channel input alphabet is finite. Random numbers, which satisfy a condition specified by a function and its value, are used to construct the stochastic encoder. The proof of the theorem is based on the hash property of an ensemble of functions, where the results are extended to a general channel by deriving an alternative formula for the capacity. Since an ensemble of sparse matrices has a hash property, we can construct a code by using sparse matrices.

The aim of this paper is to investigate relationship between inner regions for broadcast channel codes transmitting common and private messages. One of the regions is known as the Marton inner region, another region was derived by Gel'fand and Pinsker, and yet another is obtained from the saturation property and collision-resistance property. Although at first glance the Marton inner region appears to be larger than the other regions, they are in fact equivalent.

There is a flaw in the statement of Lemma 5 in the above titled paper (ibid., vol. 56, no. 5, pp. 2143-2167, May 2010), which is used in the proof of Ths. 4, 6, and 7 in the same paper. Revisions are presented here.

An LDPC matrix is used as a local encoding kernel at each link to construct a universal code to address network coding problems. It is also shown that at each terminal node the global encoding kernel that constructs a decoder becomes an LDPC matrix. This provides the perspective that decoding complexity can be reduced to a linear order of a block length by using an efficient decoding algorithm such as the sum-product algorithm.

We investigate the problem of (independent) uniform random number generation and secret key agreement. A generator for the (independent) uniform random numbers is constructed by using a sparse matrix and it is applied to a secret key agreement protocol with strong secrecy. It is proved that the rate of the proposed codes can achieve the fundamental limits.

The aim of this paper is to prove coding theorems for the wiretap channel and the secret key agreement based on the the notion of a hash property for an ensemble of functions. These theorems imply that codes using sparse matrices can achieve the optimal rate. Furthermore, fixed-rate universal coding theorems for a wiretap channel and a secret key agreement are also proved.

There are flaws in the proof of Ths. 1 and 3 in the above titled paper (ibid., vol 56, no. 6, pp. 2688-2698, Jun. 2010). Corrections are provided here.

A strongly secure wiretap channel code is proposed. The construction is based on the strong hash property introduced in Proc. ISIT2010, pp. 575-579. Since an ensemble of sparse matrices satisfies the conditions for the strong hash property, the rate of the proposed code using sparse matrices can achieve the secrecy capacity.

The aim of this paper is to introduce the construction of codes for multiple access channels based on the the notion of the strong hash property introduced in Proc. ISIT2010, pp. 575-579. Since an ensemble of sparse matrices has a strong hash property, we can use sparse matrices for code constructions.

The aim of this paper is to construct a code for broadcast channel (independent messages and no common message) based on the the notion of a stronger version of the hash property for an ensemble of functions. Since an ensemble of sparse matrices has a strong hash property, codes using sparse matrices can achieve an inner bound of capacity region.

The aim of this paper is to prove the achievability of rate regions for several coding problems by using sparse matrices (with logarithmic column degree) and maximum-likelihood (ML) coding. These problems are the Gel'fand-Pinsker problem, the Wyner-Ziv problem, and the one-helps-one problem (source coding with partial side information at the decoder). To this end, the notion of a hash property for an ensemble of functions is introduced and it is proved that an ensemble of q-ary sparse matrices satisfies the hash property. Based on this property, it is proved that the rate of codes using sparse matrices and ML coding can achieve the optimal rate.

The aim of this paper is to prove the fixed-rate universal coding theorems by using the notion of the hash property. These theorems are the fixed-rate lossless universal source coding theorem and the fixed-rate universal channel coding theorem. Since an ensemble of sparse matrices (with logarithmic column degree) satisfies the hash property requirement, it is proved that we can construct universal codes by using sparse matrices.

In the above titled paper (ibid., vol. 56, no. 5, pp. 2143-2167, May 10, there is a gap in the proof of Theorem 4. The corrections are provided here.

The aim of this paper is to prove the achievability of the general (asymmetric) channel coding problem based on the hash property. Since an ensemble of q-ary sparse matrices (the maximum column weight grows logarithmically in the block length) satisfies the hash property, it is proved that the rate of codes using sparse matrices can achieve the channel capacity.

A channel code is constructed using sparse matrices for stationary memoryless channels that do not necessarily have a symmetric property like a binary symmetric channel. It is also shown that the constructed code has the following remarkable properties: 1) Joint source-channel coding: Combining with lossy source code, which is also constructed by sparse matrices, a simpler joint source-channel code can be constructed than that constructed by the ordinary block code. 2) Universal coding: The constructed channel code has a universal property under a specified condition.

The aim of this paper is to prove the achievability of the Wyner-Ziv source coding problem by using sparse matrices and maximal-likelihood (ML) coding. To this end, the notion of a hash property for an ensemble of functions is introduced. For example, an ensemble of q-ary sparse matrices satisfies the hash property. Based on this property, it is proved that the rate of codes using sparse matrices and maximal-likelihood (ML) coding can achieve the optimal rate.

An operation that permutes symbols in a sequence is used for several coding algorithms. The probability distribution of the permuted source is investigated, where the permutation is selected at random with a uniform distribution. Moreover, the entropy of the permuted source and the mutual information shared by the original and permuted sources are derived.

Research into applying LDPC code theory, which is used for channel coding, to source coding has received a lot of attention in several research fields such as Distributed Source Coding. In this paper a source coding problem with a fidelity criterion is considered. Matsunaga et al. [6] constructed a lossy code under the conditions of a binary alphabet, a uniform distribution, and a Hamming measure of fidelity criterion. We extend their results and construct a lossy code under the extended conditions of a binary alphabet, a distribution that is not necessarily uniform, and a fidelity measure that is bounded and additive and show that the code can achieve the optimal rate, rate-distortion function.

An operation that permutes symbols in a sequence is used for several coding algorithms. The probability distribution of the permuted source is investigated, where the permutation is selected at random with a uniform distribution. Moreover, the entropy of the permuted source and the mutual information shared by the original and permuted sources are derived.

Secret key agreement is a procedure for agreeing on a secret key by exchanging messages over a public channel when a sender, a legitimate receiver and an eavesdropper have access to correlated sources. Maurer, IEEE Trans. Inf. Theory 1993 defined secret key capacity, which is the least upper bound of the key generation rate of the secret key agreement, and presented an upper and a lower bound for the secret key capacity. In this paper, the advantage distillation capacity is introduced and it is shown that this quantity equals to the secret key capacity. A naive information theoretical expression of the secret key capacity and the advantage distillation capacity is also presented. An example of correlated sources, for which an analytic expression of the secret key capacity can be obtained, is also presented

The capacity for secret key agreement for permutation-invariant and symmetric sources under a sampling attack is investigated. The supremum of the normalized secret key capacity is introduced, where the supremum is taken over all permutation-invariant sources or all symmetric sources and the normalized secret key capacity is the secret key capacity divided by the description length of the symbol. It is proved that the supremum of the normalized secret key capacity bound under a sampling attack is close to 1/m for permutation-invariant sources and O(1/m) for symmetric sources, where and m is the number of Eve's sources

This paper considers the capacity for secret key agreement under a sampling attack. Given the number of the eavesdropper's sources, we evaluate the secret key capacity bound which is defined as the supremum of the secret key capacity divided by the description length of the alphabet, where the supremum is taken over a set of probability distributions corresponding to the correlated sources. In particular, we consider symmetric sources and permutation-invariant sources. We derive inequalities which show the scaling of the secret key capacity bound with the number of the eavesdropper's sources

Linear codes for a coding problem of correlated sources are considered. It is proved that we can construct codes by using low-density parity-check (LDPC) matrices with maximum-likelihood (or typical set) decoding. As applications of the above coding problem, a construction of codes is presented for multiple-access channel with correlated additive noises and a coding theorem of parity-check codes for general channels is proved.

A scheme of secret key agreement from correlated random numbers is described. It is proved that there is a pair of sparce matrices that yields a secret key agreement in the situation wherein a sender, a legitimate receiver, and an eavesdropper have access to correlated random numbers.

The paper considers linear matrices for a coding problem for multiple access networks. It is proved that we can construct codes by using sparse matrices, which are also called low density parity check (LDPC) matrices.

Bounds on the redundancy of the recency-rank and block-sorting universal lossless data compression algorithms for finite-length sequences are presented. These algorithms are asymptotically optimal for infinite-length sequences, stationary ergodic sources in the almost-sure sense, and asymptotically mean stationary sources in the average and almost-sure sense.

Source coding theorems for general sources are presented. For a source /spl mu/, which is assumed to be a probability measure on all strings of an infinite-length sequence with a finite alphabet, the notion of almost-sure sup entropy rate is defined; it is an extension of the Shannon entropy rate. When both an encoder and a decoder know that a sequence is generated by /spl mu/, the following two theorems can be proved: (1) in the almost-sure sense, there is no variable-rate source coding scheme whose coding rate is less than the almost-sure sup entropy rate of /spl mu/, and (2) in the almost-sure sense, there exists a variable-rate source coding scheme whose coding rate achieves the almost-sure sup entropy rate of /spl mu/.