The coding problem for wiretap channels (WTCs) with causal and/or non-causal channel state information (CSI) available at the encoder (Alice) and/or the decoder (Bob) is studied, particularly focusing on achievable secret-message secret-key (SM-SK) rate pairs under the semantic security criterion. One of our main results is summarized as Theorem 3 on causal inner bounds for SM-SK rate pairs, which follows immediately by leveraging the unified seminal theorem for WTCs with non-causal CSI at Alice that has been recently established by Bunin et al.. The only thing to do here is just to re-interpret the latter non-causal scheme in a causal manner by restricting the range of auxiliary random variables appearing in non-causal encoding to a subclass of auxiliary random variables for the causal encoder. This technique is referred to as “plugging.” Then, we are able to dispense with the block-Markov encoding scheme used in the previous works by Chia and El Gamal, Fujita, and Han and Sasaki and then extend all the known results on achievable rates. The other main results include the exact SM-SK capacity region for WTCs with non-causal CSI at “both” Alice and Bob (Theorem 2), a “tighter” causal SM-SK outer bound for state-reproducing coding schemes with CSI at Alice (Proposition 4), and the exact SM-SK capacity region for degraded WTCs with causal/non-causal CSI at both Alice and Bob (Theorem 4).

The isomorphism problem in the ergodic theory is revisited from the perspective of information spectrum approach, an approach that has been developed to investigate coding problems for non-ergodic random processes in information theory. It is proved that the information spectrum is invariant under isomorphisms. This result together with an analysis of information spectrum provide a conceptually simple proof of the result by Sujan, which claims that the entropy spectrum is invariant under isomorphisms. It is also discussed under what circumstances the same information spectrum implies the existence of an isomorphism.

The problem of exactly generating a general random process (target process) by using another general random process (coin process) is studied. The performance of the interval algorithm, introduced by Han and Hoshi, is analyzed from a perspective of information spectral approach. When either the coin process or the target process has one point spectrum, asymptotic optimality of the interval algorithm among any random number generation algorithms is proved, which demonstrates utility of the interval algorithm beyond the ergodic process. The feasibility condition of exact random number generation is also elucidated.

The coding problem for wiretap channels with causal channel state information available at the encoder and/or the decoder is studied under the strong secrecy criterion. This problem consists of two aspects: one is due to wiretap channel coding and the other is due to one-time pad cipher based on the secret key agreement between Alice and Bob using the channel state information. These two aspects are closely related to each other and give rise to an intriguing tradeoff between exploiting the state to boost secret-message rates versus extracting cryptographic key to improve secrecy capabilities. This issue has yet to be understood how to optimally reconcile the two. We newly devised the “iterative” forward-backward coding scheme, combining wiretap channel coding and secret-key-agreement-based one-time pad cipher. We then established reasonable lower bounds of the secrecy capacity for wiretap channels with causal channel state information available only at the encoder (Theorem 1), which can be easily extended to general cases with various kinds of correlated channel state information at the encoder (Alice), decoder (Bob), and wiretapper (Eve). In particular, for degraded wiretap channels, we give the secret-message (secret-key) capacity bounds (Theorems 2, 4, and 5).

The problem of channel resolvability, where a given output probability distribution over a channel is approximated by encoding uniform random number as a channel input, is addressed. The channel resolvability has recently been generalized to the variable-length setting, where the variable-length uniform random number instead of the fixed-length one is encoded. Though the optimum resolvability rate can be reduced compared with the fixed-length resolvability, it is not yet clear how much resolvability rate can be saved even when the given source and channel are stationary and memoryless. Given a stationary memoryless source and a discrete memoryless channel, this paper establishes a single-letter formula for the variable-length resolvability under the variational distance as an approximation measure. When the channel is a full-rank discrete memoryless channel, the established formula reduces to a further simpler formula characterized by the mutual information between the source and the channel. The established formula also recovers a known formula for the variable-length source resolvability.

In the problem of variable-length δ -channel resolvability, the channel output is approximated by encoding a variable-length uniform random number under the constraint that the variational distance between the target and approximated distributions should be within a given constant δ asymptotically. In this paper, we assume that the given channel input is a mixed source whose components may be general sources. To analyze the minimum achievable length rate of the uniform random number, called the δ -resolvability, we introduce a variant problem of the variable-length δ -channel resolvability. A general formula for the δ -resolvability in this variant problem is established for a general channel. When the channel is an identity mapping, it is shown that the δ -resolvability in the original and variant problems coincide. This relation leads to a direct derivation of a single-letter formula for the δ -resolvability when the given source is a mixed memoryless source. We extend the result to the second-order case. As a byproduct, we obtain the first-order and second-order formulas for fixed-to-variable length source coding allowing error probability up to δ.

The coding problem for wiretap channels with causal state information available at the encoder is studied. In particular, we address the wiretap channel only with one-time state information (instead of the usual causal state information up to present) in the sense that the one-time encoder uses only the current state information Sk at each time k to establish the secrecy capacity formula under the δ-strong secrecy criterion. The coding problem for wiretap channels with one-time channel state information available at the encoder under cost constraints is also studied and lower bounds on the “δ-strong” secrecy capacity given cost are also demonstrated.

The first- and second-order optimum achievable exponents in the simple hypothesis testing problem are investigated. The optimum achievable exponent for type II error probability, under the constraint that the type I error probability is allowed asymptotically up to ε, is called the ε-optimum exponent. In this paper, we first give the second-order ε-exponent in the case where the null hypothesis and the alternative hypothesis are a mixed memoryless source and a stationary memoryless source, respectively. We next generalize this setting to the case where the alternative hypothesis is also a mixed memoryless source. We address the first-order ε-optimum exponent in this setting.

We introduce the problem of variable-length source resolvability, where a given target probability distribution is approximated by encoding variable-length uniform random numbers, and the asymptotically minimum average length rate of the uniform random numbers, called the (variable-length) resolvability, is investigated. We first analyze the variable-length resolvability with the variational distance as an approximation measure. We then extend the analysis to the case under the divergence as an approximation measure. When the asymptotically exact approximation is required, it is shown that the resolvability under the two kinds of approximation measures coincides. We also analyze the second-order variable-length resolvability.

This paper investigates the first- and second-order maximum achievable rates of codes with/without cost constraints for mixed channels whose channel law is characterized by a general mixture of (at most) uncountably many stationary and memoryless discrete channels. These channels are referred to as mixed memoryless channels with general mixture and include the class of mixed memoryless channels of finitely or countably memoryless channels as a special case. For the mixed memoryless channels with general mixture, the first-order coding theorem which gives a formula for the ε-capacity is established, and then a direct part of the second-order coding theorem is provided. A subclass of mixed memoryless channels whose component channels can be ordered according to their capacity is introduced, and the first- and second-order coding theorems are established. It is shown that the established formulas reduce to several known formulas for restricted scenarios.

This paper studies the first- and second-order maximum achievable rates of codes with/without cost constraints for general mixed channels whose channel law is characterized by a mixture of uncountably many stationary and memoryless discrete channels. These channels are referred to as general mixed memoryless channels and include mixed memoryless channels of finitely or countably many memoryless channels as a special case. For general mixed memoryless channels, the first-order coding theorem which gives a formula for the ε-capacity is established, and then a direct part of the second-order coding theorem is provided. A subclass of general mixed memoryless channels whose component channels can be ordered according to their capacity is introduced, and the first- and second-order coding theorems are established. It is shown that the established formulas reduce to several known formulas for restricted scenarios.

The wiretap channel has been devised and studied first by Wyner, and subsequently extended to the case with nondegraded general wiretap channels by Csiszár and Körner. Focusing mainly on the stationary memoryless channel with cost constraint, we newly introduce the notion of reliability and secrecy functions as a fundamental tool to analyze and/or design the performance of an efficient wiretap channel system, including binary symmetric wiretap channels, Poisson wiretap channels, and Gaussian wiretap channels. Compact formulas for those functions are explicitly given for stationary memoryless wiretap channels. It is also demonstrated that, based on such a pair of reliability and secrecy functions, we can control the tradeoff between reliability and secrecy (usually conflicting), both with exponentially decreasing rates as block length \(n\) becomes large. Four ways to do so are given on the basis of rate shifting, rate exchange, concatenation, and change of cost constraint. In addition, the notion of the \(\delta \) secrecy capacity is defined and shown to attain the strongest secrecy standard among others. The maximized versus averaged secrecy measures is also discussed.

The second-order achievable rate region in Slepian-Wolf source coding systems is investigated. The concept of second-order achievable rates, which enables us to make a finer evaluation of achievable rates, has already been introduced and analyzed for general sources in the single-user source coding problem. Analogously, in this paper, we first define the second-order achievable rate region for the Slepian-Wolf coding system to establish the source coding theorem in the second-order sense. The Slepian-Wolf coding problem for correlated sources is one of typical problems in the multiterminal information theory. In particular, Miyake and Kanaya, and Han have established the first-order source coding theorems for general correlated sources. On the other hand, in general, the second-order achievable rate problem for the Slepian-Wolf coding system with general sources remains still open up to present. In this paper, we present the analysis concerning the second-order achievable rates for general sources, which are based on the information spectrum methods developed by Han and Verdú. Moreover, we establish the explicit second-order achievable rate region for independently and identically distributed (i.i.d.) correlated sources with countably infinite alphabets and mixtures of i.i.d. correlated sources, respectively, using the relevant asymptotic normality.

The second-order achievable rate region in Slepian-Wolf source coding systems is investigated. The concept of second-order achievable rates, which enables us to make a finer evaluation of achievable rates, has already been introduced and analyzed for general sources in the single-user source coding problem. Accordingly, in this paper, we first define the second-order achievable rate region for the Slepian-Wolf coding system and establish the source coding theorem for general sources in the second-order sense. Moreover, we compute the explicit second-order achievable rate region for i.i.d. correlated sources with countably infinite alphabets and mixed correlated sources, respectively, using the relevant asymptotic normality.

The second-order achievable asymptotics in typical random number generation problems such as resolvability, intrinsic randomness, and fixed-length source coding are considered. In these problems, several researchers have derived the first-order and the second-order achievability rates for general sources using the information spectrum methods. Although these formulas are general, their computations are quite hard. Hence, an attempt to address explicit computation problems of achievable rates is meaningful. In particular, for i.i.d. sources, the second-order achievable rates have earlier been determined simply by using the asymptotic normality. In this paper, we consider mixed sources of two i.i.d. sources. The mixed source is a typical case of nonergodic sources and whose self-information does not have the asymptotic normality. Nonetheless, we can explicitly compute the second-order achievable rates for these sources on the basis of two-peak asymptotic normality. In addition, extensions of our results to more general mixed sources, such as a mixture of countably infinite i.i.d. sources or Markov sources, and a continuous mixture of i.i.d. sources, are considered.

The second-order achievable rates in typical random number generation problems are considered. In these problems, several researchers have derived the first-order and the second-order achievability rates for general sources using the information spectrum methods. Although these formulas are general, their computation are quite hard. Hence, an attempt to address explicit computation problems of achievable rates is meaningful. In this paper, we consider mixed sources of two i.i.d. sources and compute the second-order achievable rates explicitly.

The problem of network coding for multicasting a single source to multiple sinks has first been studied by Ahlswede, Cai, Li, and Yeung in 2000, in which they have established the celebrated max-flow min-cut theorem on nonphysical information flow over a network of independent channels. On the other hand, in 1980, Han has studied the case with multiple correlated sources and a single sink from the viewpoint of polymatroidal functions in which a necessary and sufficient condition has been demonstrated for reliable transmission over the network. This paper presents an attempt to unify both cases, which leads to establish a necessary and sufficient condition for reliable transmission over a network for multicasting multiple correlated sources to multiple sinks. Here, the problem of separation of source coding and network coding is also discussed.

Information theory has several traditional folklore problems about data compression or channel coding with reference to random number generation problems. Here, we focus on and reasonably formulate one of them from the viewpoint of information spectra. Specifically, we verify the validity of the folklore that the output from any source encoder working at the optimal coding rate with asymptotically vanishing probability of error looks like almost completely random.

It is shown that the idea of the successive refinement of interval partitions, which plays the key role in the interval algorithm for random number generation proposed by Han and Hoshi (see ibid., vol.43, p.599-611, 1997) is also applicable to the homophonic coding. An interval algorithm for homophonic coding is introduced which produces an independent and identically distributed (i.i.d.) sequence with probability p. Lower and upper bounds for the expected codeword length are given. Based on this, an interval algorithm for fixed-to-variable homophonic coding is established. The expected codeword length per source letter converges to H(X)/H(p) in probability as the block length tends to infinity, where H(X) is the entropy rate of the source X. The algorithm is asymptotically optimal. An algorithm for fixed-to-fixed homophonic coding is also established. The decoding error probability tends to zero as the block length tends to infinity. Homophonic coding with cost is generally considered. The expected cost of the codeword per source letter converges to c~H(X)/H(p) in probability as the block length tends to infinity, where, c~ denotes the average cost of a source letter. The main contribution of this paper can be regarded as a novel application of Elias' coding technique to homophonic coding. Intrinsic relations among these algorithms, the interval algorithm for random number generation and the arithmetic code are also discussed.

Given a general source X={X/sup n/}/sub n=1//sup /spl infin//, source coding is characterized by a pair (/spl phi//sub n/, /spl psi//sub n/) of encoder /spl phi//sub n/, and decoder /spl psi//sub n/, together with the probability of error /spl epsi//sub n//spl equiv/Pr{/spl psi//sub n/(/spl phi//sub n/(X/sup n/))/spl ne/X/sup n/}. If the length of the encoder output /spl phi//sub n/(X/sup n/) is fixed, then it is called fixed-length source coding, while if the length of the encoder output /spl phi//sub n/(X/sup n/) is variable, then it is called variable-length source coding. Usually, in the context of fixed-length source coding the probability of error /spl epsi//sub n/ is required to asymptotically vanish (i.e., lim/sub n/spl rarr//spl infin///spl epsi//sub n/=0), whereas in the context of variable-length source coding the probability of error /spl epsi//sub n/ is required to be exactly zero (i.e., /spl epsi//sub n/=0/spl forall/n=1, 2, ...). In contrast to these, we consider the problem of variable-length source coding with asymptotically vanishing probability of error (i.e., lim/sub n/spl rarr//spl infin///spl epsi//sub n/=0), and establish several fundamental theorems on this new subject.

We address variable-length intrinsic randomness problems (in the sense of Vembu and Verdu (1995)) for countably infinite source alphabet /spl chi/ under the (unnormalized) divergence distance, the normalized conditional divergence distance, and the variational distance. It turns out that under all three kinds of approximation measures the variable-length intrinsic randomness still takes the same value, called the inf-entropy rate of the source.

The reliability function problems with fixed-length source coding for the general source are studied for all rates R. Our fundamental philosophy in doing so is to convert all of the reliability function problems to the pertinent computation problems in the large derivation-probability theory. It turns out that this kind of new methodology, which was previously developed by Han (see ibid., vol.43, p.1145-64, 1997), enables us to establish quite compact general formulas of the reliability function for general sources including all nonstationary and/or nonergodic sources with countably infinite alphabet. Such general formulas are presented from the information-spectrum point of view.

The asymptotically optimal hypothesis testing problem, with general sources as the null and alternative hypotheses, is studied under exponential-type error constraints on the first kind of error probability. Our fundamental philosophy is to convert all of the hypothesis testing problems to the pertinent computation problems in the large deviation-probability theory. This methodologically new approach enables us to establish compact general formulas of the optimal exponents of the second kind of error and correct testing probabilities for the general sources including all nonstationary and/or nonergodic sources with arbitrary abstract alphabet (countable or uncountable). These general formulas are presented from the information-spectrum point of view.

We show that an optimal source code with a cost function for code symbols can be regarded as a random number generator generating a random sequence (not necessarily a sequence of fair coin bits) as the target distribution in the sense that the normalized conditional divergence between the distribution of the generated codeword distribution and the target distribution vanishes as the block length tends to infinity.

It is shown that the set of deterministic random sequences (of symbols from a finite alphabet) with respect to a computable probability measure /spl mu/, in Martin-Lof's (1966) sense, and the set of deterministic random sequences with respect to another computable probability measure /spl nu/ are disjoint if /spl mu/ and /spl nu/ are different and the measures are either i.i.d. or homogeneous finite-order irreducible Markov measures.

The paper deals with the capacity problems for the general multiple-access channel, where the channel transition probabilities are arbitrary for every blocklength n. The approach used here, which is called the information-spectrum approach, is quite different from the standard typical-sequence and/or AEP techniques. The general formula for the capacity region for the general multiple-access channel is thus established. In order to examine the potentiality, we apply it to the mixed channel to obtain the formula for its capacity region. The case where input cost constraints are imposed is also considered.

This paper presents a survey of the literature on the information-theoretic problems of statistical inference under multiterminal data compression with rate constraints. Significant emphasis is put on problems: (1) multiterminal hypothesis testing, (2) multiterminal parameter estimation and (3) multiterminal pattern classification, in either case of positive rates or zero rates. In addition, the paper includes three new results, i.e., the converse theorems for all problems of multiterminal hypothesis testing, multiterminal parameter estimation, and multiterminal pattern classification at the zero rate.

In this paper we study universal coding problems for the integers, in particular, establish rather tight lower and upper bounds for the Elias omega code and other codes. In these bounds, the so-called log-star function plays a central role. Furthermore, we investigate unbounded search trees induced by these codes, including the Bentley-Yao search tree. We will reveal beautiful recursion structures latent in these search trees as well as in these codes. Finally, we introduce the modified log-star function to reveal the existance of better prefix codes than the Elias omega code and other known codes.

The rate-distortion problem for the general class of nonstationary and/or nonergodic sources with an arbitrary distortion measure (not necessarily additive) is studied. We are especially concerned with the case of variable-rate coding under maximum-distortion criterion. It turns out that, in the framework where we cannot readily invoke the standard asymptotic equipartition property, an information-spectrum approach devised by Han and Verdu (1993) plays the key role in establishing such a general formula. Comparisons with the rate-distortion formulas with fixed-rate coding of Steinberg and Verdu (see ibid., vol.42, no.1, p.63-86, 1996) are also discussed to obtain an insight into the general features of this kind of nonstationary and nonergodic problems.

The problem of generating a random number with an arbitrary probability distribution by using a general biased M-coin is studied. An efficient and very simple algorithm based on the successive refinement of partitions of the unit interval (0, 1), which we call the interval algorithm, is proposed. A fairly tight evaluation on the efficiency is given. Generalizations of the interval algorithm to the following cases are investigated: (1) output sequence is independent and identically distributed (i.i.d.); (2) output sequence is Markov; (3) input sequence is Markov; (4) input sequence and output sequence are both subject to arbitrary stochastic processes.

The (noiseless) fixed-length source coding theorem states that, except for outcomes in a set of vanishing probability, a source can be encoded at its entropy but not more efficiently. It is well known that the asymptotic equipartition property (AEP) is a sufficient condition for a source to be encodable at its entropy. This paper shows that the AEP is necessary for the source coding theorem to hold for nonzero-entropy finite-alphabet sources. Furthermore, we show that a nonzero-entropy finite-alphabet source satisfies the direct coding theorem if and only if it satisfies the strong converse. In addition, we introduce the more general setting of nonserial information sources which need not put out strings of symbols. In this context, which encompasses the conventional serial setting, the AEP is equivalent to the validity of the strong coding theorem. Fundamental limits for data compression of nonserial information sources are shown based on the flat-top property-a new sufficient condition for the AEP.

A new type of sufficient condition is provided for a probability distribution on the nonnegative integers to be given an optimal D-ary prefix code by a Huffman-type algorithm. In the justification of our algorithm, we introduce two new (essentially one) concepts as the definition of the "optimality" of a prefix D-ary code, which are shown to be equivalent to that defined in the traditional way. These new concepts of the optimality are meaningful even for the case where the Shannon entropy H(P) diverges.

The multiterminal estimation theory deals with an entirely novel problem which takes place in the void between information theory and statistics, that is, what amount of Fisher information can be attained under a restriction on the amount of Shannon information. The key idea here is the indivisible fusion of the information-theoretic universal coding problem and the statistical maximum-likelihood parameter estimation problem. The main result is the explicit establishment of maximum-likelihood estimators attainable under the rate-constrained universal coding scheme, which is shown to have a variance equal to the inverse of the Fisher information. This may be regarded as giving a multiterminal generalization of the usual Cramer-Rao bound. Relevant properties and examples of these maximum-likelihood estimators are also shown.

We consider the Bayesian parameter estimation problem where the value of a finitary parameter X should be decided on the basis of i.i.d. sample Y/sup n/ of size n. In this context, the amount of missing information on X after observing Y/sup n/ may be evaluated by the posterior entropy, which is often called the equivocation or the conditional entropy, of X given Y/sup n/, while it is well known that the minimum possible probability of error in estimating X is achieved by the maximum a posteriori probability (MAP) estimator. In this work, the focus is on the asymptotic relation between the posterior entropy and the MAP error probability as the sample size n becomes sufficiently large. It is shown that if the sample size n is large enough, the posterior entropy as well as the MAP error probability decay with n to zero at the identical exponential rate, and that the maximum achievable exponent for this decay is determined by the minimum Chernoff information over all the possible pairs of distinct parameter values.

The Fano inequality gives a lower bound on the mutual information between two random variables that take values on an M-element set, provided at least one of the random variables is equiprobable. The authors show several simple lower bounds on mutual information which do not assume such a restriction. In particular, this ran be accomplished by replacing log M with the infinite-order Renyi entropy in the Fano inequality. Applications to hypothesis testing are exhibited along with bounds on mutual information in terms of the a priori and a posteriori error probabilities.>

Universal coding for the Slepian-Wolf (1973) data compression system is considered. We shall demonstrate based on a simple observation that the error exponent given by Csiszar and Korner (1980) for the universal coding system can strictly be sharpened in general for a region of relatively higher rates. This kind of observation can be carried over also to the case of lower rates outside the Slepian-Wolf region, which establishes the strong converse along with the optimal exponent.>

A formula for the capacity of arbitrary single-user channels without feedback (not necessarily information stable, stationary, etc.) is proved. Capacity is shown to equal the supremum, over all input processes, of the input-output inf-information rate defined as the liminf in probability of the normalized information density. The key to this result is a new converse approach based on a simple new lower bound on the error probability of m-ary hypothesis tests among equiprobable hypotheses. A necessary and sufficient condition for the validity of the strong converse is given, as well as general expressions for /spl epsiv/-capacity.>

Given a channel and an input process with output statistics that approximate the original output statistics with arbitrary accuracy, the randomness of the input processes is studied. The notion of resolvability of a channel, defined as the number of random bits required per channel use in order to generate an input that achieves arbitrarily accurate approximation of the output statistics for any given input process, is introduced. A general formula for resolvability that holds regardless of the channel memory structure is obtained. It is shown that for most channels, resolvability is equal to the Shannon capacity. By-products of the analysis are a general formula for the minimum achievable source coding rate of any finite-alphabet source and a strong converse of the identification coding theorem, which holds for any channel that satisfies the strong converse of the channel coding theorem.>

The identification capacity is the maximal iterated logarithm of the number of messages divided by the blocklength that can be reliably transmitted when the receiver is only interested in deciding whether a specific message was transmitted or not. The identification coding theorem of R. Ahlswede and G. Dueck (1989) for single-user discrete memoryless channels states that the identification capacity is equal to the Shannon capacity. A novel method to prove the converse to the identification coding theorem is shown to achieve the strong version of the result. Identification plus transmission (IT) coding, a variant of the original problem of identification via channels, is proposed in the context of a common problem in point-to-multipoint communication, where a central station wishes to transmit information reliably to one of N terminals, whose identity is not predetermined. The authors show that as long as log log N is smaller than the number of bits to be transmitted, IT codes allow information transmission at channel capacity.>

A certain class of variable-length codes with feedback whose codeword lengths are uniformly upper bounded is considered. For this class of variable-length codes, it is shown that the zero-error capacity region for the, single-user and multiuser channels with feedback can be extended up to the ordinary average-error capacity under some conditions, if variable-length codes (semiblock codes) are used in place of fixed-length codes. This condition is different from that of M.V. Burnashev (1976) for variable-length codes with feedback but without any uniform bound on the codeword lengths. It is also shown that the capacity region for variable-length feedback codes coincides with that for fixed-length feedback codes.>

A statistical inference problem for a two-terminal information source emitting mutually correlated signals X and Y is treated. Let sequences X/sup n/ and Y/sup n/ of n independent observations be encoded independently of each other into message sets M/sub X/ and M/sub Y/ at rates R/sub 1/ and R/sub 2/ per letter, respectively. This compression causes a loss of the statistical information available for testing hypotheses concerning X and Y. The loss of statistical information is evaluated as a function of the amounts R/sub 1/ and R/sub 2/ of the Shannon information. A complete solution is given in the case of asymptotically complete data compression, R/sub 1/, R/sub 2/ to 0 as n to infinity . It is shown that the differential geometry of the manifold of all probability distributions plays a fundamental role in this type of multiterminal problem connecting Shannon information and statistical information. A brief introduction to the dually coupled e-affine and m-affine connections together with e-flatness and m-flatness is given.>

Multiterminal hypothesis testing is considered, subject to the exponential-type constraint alpha /sub n/>

The authors present the answer to a so-called converse problem by giving the explicit form of the power exponent. This strong converse should be regarded as completing a weak converse previously obtained by R.E. Blahut (1974).>

The multiterminal hypothesis testingH: XYagainstH̄: X̄Ȳis considered whereX^{n} (X̄^{n})andY^{n} (Ȳ^{n})are separately encoded at ratesR_{1}andR_{2}, respectively. The problem is to determine the minimum\beta_{n}of the second kind of error probability, under the condition that the first kind of error probability\alpha_{n} \leq \epsilonfor a prescribed0 < \epsilon < 1. A good lower bound\theta_{L}(R_{1}, R_{2})on the power exponent\theta (R_{1}, R_{2},\epsilon)= \lim \inf_{n \rightarrow \infty}(-1/n \log \beta_{n})is given and several interesting properties are revealed. The lower bound is tighter than that of Ahlswede and Csiszár. Furthermore, in the special case of testing against independence, this bound turns out to coincide with that given by them. The main arguments are devoted to the special case withR_{2} = \inftycorresponding to full side information forY^{n}(Ȳ^{n}). In particular, the compact solution is established to the complete data compression cases, which are useful in statistics from the practical point of view.

A new coding theorem for the broadcast channel with arbitrarily correlated sources is presented. The result covers all the previously established coding techniques. In particular, it includes Marton's coding theorem as a properly special case.

Consider separate encoding of correlated sourcesX^{n}=(X_{l}, \cdots ,X_{n}), Y^{n} = (Y_{l}, \cdots ,Y_{n})for the decoder to reliably reproduce a function\{F(X_{i}, Y_{i})\}^{n}_{i=1}. We establish the necessary and sufficient condition for the set of all achievable rates to coincide with the Slepian-Wolf region whenever the probability densityp(x,y)is positive for all(x,y).

A general coding scheme for the nonrestricted memoryless discrete two-way channel is presented based on the introduction of auxiliary random variables forming a stationary Markov process. The coding scheme yields an achievable rate region which exceeds the inner bound of Shannon in the general case. A finite cardinality bound for the auxiliary random variables is given, showing that the region is computable. Finally, the capacity region for the memoryless Gaussian two-way channel is established.

A simple proof of the coding theorem for the multiple-access channel (MAC) with arbitrarily correlated sources (DMCS) of Cover-El Carnal-Salehi, which includes the results of Ahlswede for the MAC and of Slepian-Wolf for the DMCS and the MAC as special cases, is first given. A coding theorem is introduced and established for another type of source-channel matching problem, i.e., a system of source coding with side information via a MAC, which can be regarded as an extension of the Ahlswede-Körner-Wyner type noiseless coding system. This result is extended to a more general system with several principal sources and several side information sources subject to cross observation at the encoders in the sense of Han. The regions are shown to be optimal in special situations. Dueck's example shows that this is in general not the case for the result of Cover-El Gamal-Salehi and the present work. In another direction, the achievable rate region for the module-two sum source network found by Körner-Marton is improved. Finally, some ideas about a new approach to the source-channel matching problem in multi-user communication theory are presented. The basic concept is that of a correlated channel code. The approach leads to several new coding problems.

The deterministic broadcast channel with two output terminals is studied for the case of a common message. The capacity region for this channel is established by means of a random coding argument combining the standard channel coding technique and the standard source coding technique.

A new achievable rate region for the general interference channel which extends previous results is presented and evaluated. The technique used is a generalization of superposition coding to the multivariable case. A detailed computation for the Gaussian channel case clarifies to what extent the new region improves previous ones. The capacity of a class of Gaussian interference channels is also established.

A unified treatment of a large class of multiterminal noiseless source coding problems including all previously studied situations is presented. A unified achievable rate region is established for this class by a coding technique based on the typical sequence criterion. This region is tight for all the previously studied situations.