This paper characterizes the fundamental limits on overflow probability for variable-length codes with codeword cost using the smooth Rényi entropy approach. For general sources, we establish precise finite-length bounds, proving that the optimal achievable overflow threshold R(n, ε) is bounded by using the smooth max entropy. Conversely, we demonstrate that unachievable overflow thresholds are characterized by smooth min entropy. We extend the achievable threshold results to the asymptotic case, proving convergence to the smooth max entropy rate. Our work establishes a direct connection between overflow probability bounds and smooth Rényi entropy, providing a comprehensive theoretical framework for analyzing variable-length codes under cost constraints.

In the field of information theory, the optimum hypothesis testing exponent, which is defined as the maximum exponent of the type II error probability under the condition that the type I error probability is smaller than or equal to some constant, has been analyzed in several settings. In particular, information spectrum methods, which is one of efficient techniques in information theory, have been applied to the hypothesis testing problem, and have succeeded in giving the general formula of the optimum hypothesis testing exponent. Recently, two typical extensions of the binary hypothesis testing setting have received much attention. One is the case where we do not know the probabilistic distributions. The other is the hypothesis testing problem in the presence of noise. However, information spectrum methods have not yet been applied to the hypothesis testing in these two directions. Hence, in this paper we develop information spectrum methods to treat the hypothesis testing in these settings. In particular, we first consider the hypothesis testing with noise and show the optimum exponent of the type II error probability under the condition that the type I error probability is smaller than or equal to some constant. Then, we extend this result to the case where probability distributions of data are unknown.

The self-random number generation (SRNG) problem is considered for general setting. In the literature, the optimum SRNG rate with respect to the variational distance has been discussed. In this paper, we try to characterize the optimum SRNG rate with respect to a subclass of f-divergences. The subclass of f-divergences considered in this paper includes typical distance measures such as the variational distance, the KL divergence, the Hellinger distance and so on. Hence our result can be considered as a generalization of the previous result with respect to the variational distance. We also apply our results to the rate distortion perception function.

This paper deals with the relationship between the intrinsic randomness (IR) problem and the fixed-length source coding problem. The IR problem is one of random number generation problems and optimum achievable rates (optimum IR rate) with respect to several approximation measures such as the variational distance, the Kullback-Leibler (KL) divergence and f-divergences, have been investigated. In particular, it has been shown that the optimum IR rate with respect to the variational distance has a close relationship with the supremum of the unachievable rate in the source coding problem. Inspired by this result, in this paper, we consider the optimum IR rate with respect to a subclass of f-divergences and try to show a relationship with the unachievable rate in the source coding problem. The subclass of f-divergences considered in this paper includes several well-known measures, such as the variational distance, the KL divergence, the Hellinger distance. We also consider a class of normalized f-divergences, which includes the normalized KL divergence.

The intrinsic randomness (IR) problem is considered for general setting. In the literature, the optimum IR rate with respect to the variational distance has been characterized in two ways. One is based on the information spectrum quantity and the other is based on the smooth Rényi entropy. Recently, Nomura has revealed the optimum IR rate with respect to$f$-divergences, which includes the variational distance, the Kullback-Leibler (KL) divergence and so on, by using the informational spectrum quantity. In this paper, we try to characterize the optimum IR rate with respect to a subclass of$f$-divergences by using the smooth Min entropy. The subclass of$f$-divergences considered in this paper includes typical distance measures such as the total variational distance, the KL divergence, the Hellinger distance and so on.

The source resolvability problem (or resolvability problem for short) is one of random number generation problems in information theory. In the literature, the optimum achievable rates in the resolvability problem have been characterized in different two ways. One is based on the information spectrum quantity and the other is based on the smooth Rényi entropy. Recently, Nomura has revealed the optimum achievable rate with respect to the f-divergence, which includes the variational distance, the Kullback-Leibler (KL) divergence and so on. On the other hand, the optimum achievable rates with respect to the variational distance has been characterized by using the smooth Rényi entropy. In this paper, we try to extend this result to the case of other distances. To do so, we consider the resolvability problem with respect to the subclass of f-divergences and determine the optimum achievable rate in terms of the smooth Rényi entropy. The subclass of f-divergences considered in this paper includes typical distance measures such as the total variational distance, the KL divergence, the Hellinger distance and so on.

This paper deals with the relationship between the source resolvability problem (or resolvability problem for short) and the fixed-length source coding problem. In the literature, optimum achievable rates in the resolvability problem (optimum resolvability rate) with respect to the variational distance as well as the Kullback-Leibler (KL) divergence, have already been analyzed. The relationship between the optimum resolvability rate and the optimum rate of the fixed-length source coding has also been clarified in each cases. In particular, it has been reported that the optimum source resolvability rate with respect to the normalized KL divergence has a close relationship with the optimum fixed-length source coding rate with the correct decoding exponent. Recently, the optimum resolvability rate with respect to a class of f -divergences has been analyzed. This result can be considered as a generalization of the optimum resolvability rate with respect to the unnormalized KL divergence. However, unnormalized f-divergences has not been considered yet in the resolvability problem. Hence, in this paper, we consider the resolvability problem with respect to a class of unnormalized f-divergences. In particular, we derive the relationship between the optimum resolvability rate with a class of normalized f- divergences and the optimum rate of the fixed-length source coding.

This paper deals with two typical random number generation problems in information theory. One is the source resolvability problem (resolvability problem for short) and the other is the intrinsic randomness problem. In the literature, optimum achievable rates in these two problems with respect to the variational distance as well as the Kullback-Leibler (KL) divergence have already been analyzed. On the other hand, in this study we consider these two problems with respect to f-divergences. The f-divergence is a general non-negative measure between two probabilistic distributions on the basis of a convex function f. The class of f-divergences includes several important measures such as the variational distance, the KL divergence, the Hellinger distance and so on. Hence, it is meaningful to consider the random number generation problems with respect to f-divergences. In this paper, we impose some conditions on the function f so as to simplify the analysis, that is, we consider a subclass of f-divergences. Then, we first derive general formulas of the first-order optimum achievable rates with respect to f-divergences. Next, we particularize our general formulas to several specified functions f. As a result, we reveal that it is easy to derive optimum achievable rates for several important measures from our general formulas. The second-order optimum achievable rates and optimistic optimum achievable rates have also been investigated.

This paper deals with the source resolvability problem which is one of typical random number generation problems. In the literatures, the achievable rate in the source resolvability problem with respect to the variational distance as well as the Kullback-Leibler (KL) divergence have been analyzed. On the other hand, in this study we consider the problem with respect to a subclass of f-divergence. The f-divergence is a general non-negative measure between two probabilistic distributions and includes several important measures such as the variational distance, the KL divergence, the Hellinger distance and so on. Hence, it is meaningful to consider the source resolvability problem with respect to the f-divergence. We derive the general formula of the optimum achievable rate for a certain subclass of the f-divergence. Then, we reveal that it is easy to derive previous results from our general formula.

This paper deals with the intrinsic randomness (IR) problem, which is one of typical random number generation problems. In the literature, the optimum achievable rates in the IR problem with respect to the variational distance as well as the Kullback-Leibler (KL) divergence have already been analyzed. On the other hand, in this study we consider the IR problem with respect to a subclass of f-divergences. The f-divergence is a general non-negative measure between two probabilistic distributions and includes several important measures such as the total variational distance, the X2-divergence, the KL divergence, and so on. Hence, it is meaningful to consider the IR problem with respect to the f-divergence. In this paper, we assume some conditions on the f-divergence for simplifying the analysis. That is, we focus on a subclass of f-divergences. In this problem setting, we first derive the general formula of the optimum achievable rate. Next, we show that it is easy to derive the optimum achievable rate with respect to the variational distance, the KL divergence, and the Hellinger distance from our general formula.

Lossless variable-length source coding with codeword cost is considered forgeneralsources. The problem setting, where we impose on unequal costs on code symbols, is called the variable-length coding with codeword cost. In this problem, the infimum of average codeword cost have already been determined forgeneralsources. On the other hand, the overflow probability, which is defined as the probability of codeword cost being above a threshold, have not been considered yet. In this paper, we first determine the infimum of achievable threshold in the first-order sense and the second-order sense forgeneralsources with additive memoryless codeword cost. Then, we compute it for some special sources such as i.i.d. sources and mixed sources. A generalization of the codeword cost is also discussed.

The variable-length source coding problem allowing the error probability up to some constant is considered for general sources. In this problem, the optimum mean codeword length of variable-length codes has already been determined. On the other hand, in this paper, we focus on the overflow (or excess codeword length) probability instead of the mean codeword length. The infimum of overflow thresholds under the constraint that both of the error probability and the overflow probability are smaller than or equal to some constant is called the optimum overflow threshold. In this paper, we first derive finite-length upper and lower bounds on these probabilities so as to analyze the optimum overflow thresholds. Then, by using these bounds, we determine the general formula of the optimum overflow thresholds in both of the first-order and second-order forms. Next, we consider another expression of the derived general formula so as to reveal the relationship with the optimum coding rate in the fixed-length source coding problem. Finally, we apply the general formula derived in this paper to the case of stationary memoryless sources.

The first- and second-order optimum achievable rates in the source resolvability problem are considered for general sources. In the literature, the achievable rates in the resolvability problem with respect to the variational distance as well as the normalized Kullback-Leibler (KL) divergence have already been analyzed. On the other hand, in this study we consider the source resolvability problem with respect to (unnormalized) KL divergence and derive general formulas of the first- and second-order optimum achievable rates. Relationships with other problems in information theory have also been discussed.

The variable-length source coding with unequal cost allowing error probability is considered for general sources. In this setting, the first- and second-order optimum mean codeword cost have already been determined. On the other hand, we focus on the overflow probability of codeword cost and determine the general formulas of the first- and second-order optimum achievable overflow threshold. We also apply our general formulas to the stationary memoryless source.

The first- and second-order optimum achievable exponents in the simple hypothesis testing problem are investigated. The optimum achievable exponent for type II error probability, under the constraint that the type I error probability is allowed asymptotically up to ε, is called the ε-optimum exponent. In this paper, we first give the second-order ε-exponent in the case where the null hypothesis and the alternative hypothesis are a mixed memoryless source and a stationary memoryless source, respectively. We next generalize this setting to the case where the alternative hypothesis is also a mixed memoryless source. We address the first-order ε-optimum exponent in this setting.

The variable-length coding problem allowing the error probability up to some constant is considered for general sources. In this problem, we focus on the overflow (or excess codeword length) probability We also consider another expression of our general formula so as to reveal the relationship with the optimum coding rate in the fixed-length source coding problem.

We consider a new concept of achievability in the variable-length lossy source coding on the basis of the probability of the union of the overflow of codeword length and the excess distortion. In this setting, our main concern is to determine the achievable rate-distortion region for a given source and distortion measure. To this end, we first derive non-asymptotic upper and lower bounds, and then we derive general formulas of this achievable rate-distortion region in the first- and second-order sense. Finally, we apply our general formulas to stationary memoryless sources with an additive distortion measure.

This paper investigates the first- and second-order maximum achievable rates of codes with/without cost constraints for mixed channels whose channel law is characterized by a general mixture of (at most) uncountably many stationary and memoryless discrete channels. These channels are referred to as mixed memoryless channels with general mixture and include the class of mixed memoryless channels of finitely or countably memoryless channels as a special case. For the mixed memoryless channels with general mixture, the first-order coding theorem which gives a formula for the ε-capacity is established, and then a direct part of the second-order coding theorem is provided. A subclass of mixed memoryless channels whose component channels can be ordered according to their capacity is introduced, and the first- and second-order coding theorems are established. It is shown that the established formulas reduce to several known formulas for restricted scenarios.

This paper deals with a fixed-length lossy source coding problem with some excess distortion probability called the source coding problem with ε-fidelity criterion. In this problem, the rate-distortion function and the distortion-rate function have already been characterized for i.i.d. sources with an additive distortion measure in the sense of first-order and second-order coding rates. However, general formulas for these functions have not been revealed up to present. Hence, in this paper we derive general formulas for the rate-distortion function and the distortion-rate function with the ε-fidelity criterion in both of the first-order and second-order cases. A relationship between our general formulas and previous results are also discussed.

This paper studies the first- and second-order maximum achievable rates of codes with/without cost constraints for general mixed channels whose channel law is characterized by a mixture of uncountably many stationary and memoryless discrete channels. These channels are referred to as general mixed memoryless channels and include mixed memoryless channels of finitely or countably many memoryless channels as a special case. For general mixed memoryless channels, the first-order coding theorem which gives a formula for the ε-capacity is established, and then a direct part of the second-order coding theorem is provided. A subclass of general mixed memoryless channels whose component channels can be ordered according to their capacity is introduced, and the first- and second-order coding theorems are established. It is shown that the established formulas reduce to several known formulas for restricted scenarios.

This paper addresses two problems of variablelength coding with a fidelity criterion for general sources, in which either the probability of codeword length overflow or that of excess distortion for a given threshold is allowed up to ε. In each problem, a general formula for the achievable region of coding rates and distortion levels is established via information spectrum methods. It is shown that there is a tight connection between the two problems, and the achievable regions are the same under a mild condition on the distortion measure. As a consequence, it turns out that superficially different general formulas for the two coding problems coincide with each other.

For the class of mixed channels decomposed into stationary memoryless channels, single-letter characterizations of the ε-capacity have not been known except for restricted classes of channels such as the regular decomposable channel introduced by Winkelbauer. This paper gives single-letter characterizations of ε-capacity for mixed channels decomposed into at most countably many memoryless channels with a finite input alphabet with/without cost constraints. It is shown that the given characterization reduces to the one for the channel capacity given by Ahlswede when ε is zero. Some properties of the function of the ε-capacity are analyzed.

The second-order achievable rate region in Slepian-Wolf source coding systems is investigated. The concept of second-order achievable rates, which enables us to make a finer evaluation of achievable rates, has already been introduced and analyzed for general sources in the single-user source coding problem. Analogously, in this paper, we first define the second-order achievable rate region for the Slepian-Wolf coding system to establish the source coding theorem in the second-order sense. The Slepian-Wolf coding problem for correlated sources is one of typical problems in the multiterminal information theory. In particular, Miyake and Kanaya, and Han have established the first-order source coding theorems for general correlated sources. On the other hand, in general, the second-order achievable rate problem for the Slepian-Wolf coding system with general sources remains still open up to present. In this paper, we present the analysis concerning the second-order achievable rates for general sources, which are based on the information spectrum methods developed by Han and Verdú. Moreover, we establish the explicit second-order achievable rate region for independently and identically distributed (i.i.d.) correlated sources with countably infinite alphabets and mixtures of i.i.d. correlated sources, respectively, using the relevant asymptotic normality.

The second-order achievable rate region in Slepian-Wolf source coding systems is investigated. The concept of second-order achievable rates, which enables us to make a finer evaluation of achievable rates, has already been introduced and analyzed for general sources in the single-user source coding problem. Accordingly, in this paper, we first define the second-order achievable rate region for the Slepian-Wolf coding system and establish the source coding theorem for general sources in the second-order sense. Moreover, we compute the explicit second-order achievable rate region for i.i.d. correlated sources with countably infinite alphabets and mixed correlated sources, respectively, using the relevant asymptotic normality.

The second-order achievable asymptotics in typical random number generation problems such as resolvability, intrinsic randomness, and fixed-length source coding are considered. In these problems, several researchers have derived the first-order and the second-order achievability rates for general sources using the information spectrum methods. Although these formulas are general, their computations are quite hard. Hence, an attempt to address explicit computation problems of achievable rates is meaningful. In particular, for i.i.d. sources, the second-order achievable rates have earlier been determined simply by using the asymptotic normality. In this paper, we consider mixed sources of two i.i.d. sources. The mixed source is a typical case of nonergodic sources and whose self-information does not have the asymptotic normality. Nonetheless, we can explicitly compute the second-order achievable rates for these sources on the basis of two-peak asymptotic normality. In addition, extensions of our results to more general mixed sources, such as a mixture of countably infinite i.i.d. sources or Markov sources, and a continuous mixture of i.i.d. sources, are considered.

The second-order achievable rates in typical random number generation problems are considered. In these problems, several researchers have derived the first-order and the second-order achievability rates for general sources using the information spectrum methods. Although these formulas are general, their computation are quite hard. Hence, an attempt to address explicit computation problems of achievable rates is meaningful. In this paper, we consider mixed sources of two i.i.d. sources and compute the second-order achievable rates explicitly.

Lossless variable-length source coding with unequal cost function is considered for general sources. In this problem, the codeword cost instead of codeword length is important. The infimum of average codeword cost has already been determined for general sources. We consider the overflow probability of codeword cost and determine the infimum of achievable overflow threshold. Our analysis is on the basis of information-spectrum methods and hence valid through the general source.

We consider the source coding problem with side information. Especially, we consider the FV code in the case that the encoder and the decoder can see side information. We obtain the condition that there exists a FV code under the condition that the overflow probability is smaller than or equal to some constant.

Lossless fixed-to-variable(FV) length codes are considered. The overflow probability is one of criteria that evaluate the performance of FV code. In the single source coding problem, there were many researches on the overflow probability. Recently, the source coding problem for correlated sources, such as Slepian-Wolf coding problem or source coding problem with side information, is one of main topics in information theory. In this paper, we consider the source coding problem with side information. Especially, we consider the FV code in the case that the encoder and the decoder can see side information. In this case, several codes were proposed and their mean code lengths were analyzed. However, there was no research about the overflow probability. We shall show two lemmas about the overflow probability. Then we obtain the condition that there exists a FV code under the condition that the overflow probability is smaller than or equal to some constant.

In this paper, we generalize the achievability of variable-length coding from two viewpoints. One is the definition of an overflow probability, and the other is the definition of an achievability. We define the overflow probability as the probability of codeword length, not per symbol, is larger thanetanand we introduce theisin-achievability of variable-length codes that implies an existence of a code for the source under the condition that the overflow probability is smaller than or equal toisin. Then we show that theisin-achievability of variable-length codes is essentially equivalent to theisin-achievability of fixed-length codes for general sources. Moreover we show the condition ofisin-achievability for some restricted sources givenisin.

The authors developed an efficient high-level synthesis and optimization system for large-scale circuits, which reduces the total number of fan-ins in the technology-independent phase and adjusts speed and area after technology mapping is completed. A description is presented of multi-level logic optimization techniques based on refined weak division methods and additional functions for carrying out good optimization with only a slight overhead. The authors also describe technology mapping and local optimization techniques suitable for high-level CAD systems. The system has shown that multi-level logic optimization in VLSIs with more than 100000 gates (that is, VLSIs whose control logic comprises more than 10000 gate circuits) is possible in practical CPU time.>