Data science has been attracting a great deal of attention in recent years because of its promise to extract value from the data that abounds in society. In the control engineering field, data-driven design, in which control system design can be performed directly from data, can also be considered a part of data science. Using the data-driven approach, a control system can be optimized directly from controlled data. However, even if a system is optimally designed, its behavior cannot be verified until the system is actually controlled. Therefore, in the present study, a data-driven response estimation-based tuning (DRET) is proposed in order to design a control system based on the estimated time response. It is applied to the control system design of not only stable systems but also unstable systems. In the design of DRET, a finite impulse response filter is used and a model matching problem is solved directly from the control data, to compensate for the difference between an original objective function and a data-driven objective function. The proposed method is applied to the control system design of a ball-and-beam system, which is an unstable system, and its usefulness is verified.

This study considers the unconditional smooth Rényi entropy proposed by Renner and Wolf [ASIACRYPT, 2005], the smooth conditional Rényi entropy proposed by Kuzuoka [IEEE Trans. Inf. Th., 66(3), 1674–1690, 2020], and a novel quantity which we term theconditional smooth-⋆entropy.The latter two quantities can be specialized to the first in the absence of side-information. We explore the operational roles of these smooth Rényi entropies by establishing one-shot coding theorems for several information-theoretic problems, including Campbell’s source coding problem, the Arıkan–Massey guessing problem, and the Bunte–Lapidoth task encoding problem. We consider these problems in cases where the errors are non-vanishing and for each problem, we consider two error formalisms: the average and maximum error criteria, where the averaging and maximization are taken with respect to the side-information. Using the one-shot coding theorems, we conclude that Kuzuoka’s smooth conditional Rényi entropy and the conditional smooth-⋆ entropy are the solutions to the problems involving the average and maximum error criteria, respectively. Furthermore, we examine asymptotic expansions of these entropies when the underlying source with its side-information is stationary and memoryless. Applying our asymptotic expansions to the one-shot coding theorems, we derive various fundamental limits for these problems. We show that, under non-degenerate settings, the first-order fundamental limits differ under the average and maximum error criteria. This is in contrast to a different but related setting considered by the present authors [IEEE Trans. Inf. Th., 66(12), 7565–7587, 2020], for variable-length conditional source coding allowing errors, in which the first-order terms are identical but the second-order terms are different under these error criteria.

This study investigates the fundamental limits of variable-length compression in which prefix-free constraints are not imposed (i.e., one-to-one codes are studied) and non-vanishing error probabilities are permitted. Due in part to a crucial relation between the variable-length and fixed-length compression problems, our analysis requires a careful and refined analysis of the fundamental limits of fixed-length compression in the setting where the error probabilities are allowed to approach either zero or one polynomially in the blocklength. To obtain the refinements, we employ tools from moderate deviations and strong large deviations. Finally, we provide the third-order asymptotics for the problem of variable-length compression with non-vanishing error probabilities. We show that unlike several other information-theoretic problems in which the third-order asymptotics are known, for the problem of interest here, the third-order term depends on the permissible error probability.

Variable-length compression without prefix-free constraints and with side-information available at both encoder and decoder is considered. Instead of requiring the code to be error-free, we allow for it to have a non-vanishing error probability. We derive one-shot bounds on the optimal average codeword length by proposing two new information quantities; namely, the conditional and unconditional ε-cutoff entropies. Using these one-shot bounds, we obtain the second-order asymptotics of the problem under two different formalisms-the average and maximum probabilities of error with respect to the side-information. While the first-order terms in the asymptotic expansions for both formalisms are identical, we find that the source dispersion under the average error formalism is, in most cases, strictly smaller than its maximum counterpart. Applications to a certain class of guessing problems, previously studied by Kuzuoka [IEEE Trans. Inf. Theory, vol. 66, no. 3, pp. 1674-1690, 2020], are also discussed.

This study examines asymptotic expansions of the unconditional and conditional smooth Rényi entropies for a memoryless source. Using these smooth Rényi entropies, we establish one-shot coding theorems of several information-theoretic problems: Campbell's source coding, guessing, and task encoding problems, all allowing errors. Applying our asymptotic expansions to the derived one-shot coding theorems, we provide various asymptotic fundamental limits of these problems in the regime of non-vanishing error probabilities.

This study proposes modular arithmetic erasure channels (MAECs), a novel class of erasure-like channels with an input alphabet that need not be binary. This class contains the binary erasure channel (BEC) and some other known erasure-like channels as special cases. For MAECs, we provide recursive formulas of Arıkan-like polar transform to simulate channel polarization. In other words, we show that the synthetic channels of MAECs are equivalent to other MAECs. This is a generalization of well-known recursive formulas of the polar transform for BECs. Using our recursive formulas, we also show that a recursive application of the polar transform for MAECs results in multilevel channel polarization, which is an asymptotic phenomenon that is characteristic of non-binary polar codes. Specifically, we establish a method to calculate the limiting proportions of the partially noiseless and noisy channels that are generated as a result of multilevel channel polarization for MAECs. In the particular case of MAECs, this calculation method solves an open problem posed by Nasser (2017) in the study of non-binary polar codes.

The paper derives the optimal second-order coding rate for the continuous-time Poisson channel. We also obtain bounds on the third-order coding rate. This is the first instance of a second-order result for a continuous-time channel. The converse proof hinges on a novel construction of an output distribution induced by Wyner's discretized channel and the construction of an appropriate ϵ-net of the input probability simplex. While the achievability proof follows the general program to prove the third-order term for non-singular discrete memoryless channels put forth by Polyanskiy, several non-standard techniques-such as new definitions and bounds on the probabilities of typical sets using logarithmic Sobolev inequalities-are employed to handle the continuous nature of the channel.

Variable-length compression without prefix-free constraints and with side-information available at both encoder and decoder is considered. Instead of requiring the code to be error-free, we allow for it to have a non-vanishing error probability. We derive one-shot bounds on the optimal average codeword length by proposing two new information quantities; namely, the conditional and unconditional ε-cutoff entropies. Using these one-shot bounds, we obtain the second-order asymptotics of the problem under two different formalisms-the average and maximum probabilities of error over the realization of the side-information. While the first-order terms in the asymptotic expansions for both formalisms are identical, we find that the source dispersion under the average error formalism is, in most cases, strictly smaller than its maximum error counterpart. Applications to a certain class of guessing problems, previously studied by Kuzuoka (2020), are also discussed.

Polar transforms are central operations in the study of polar codes. This paper examines polar transforms for non-stationary memoryless sources on possibly infinite source alphabets. This is the first attempt of source polarization analysis over infinite alphabets. The source alphabet is defined to be a Polish group, and we handle the Arikan-style two-by-two polar transform based on the group. Defining erasure distributions based on the normal subgroup structure, we give recursive formulas of the polar transform for erasure distributions. We then show concrete examples of multilevel source polarization with countably infinite levels when the group is locally cyclic. We derive this result via elementary techniques in lattice theory.

The paper derives the optimal second-order coding rate for the continuous-time Poisson channel. This is the first instance of a second-order result for a continuous-time channel. The converse proof hinges on a novel construction of an output distribution induced by Wyner's discretized channel and the construction of an appropriate ε-net of the input probability simplex. An extended version of this paper is accessible at [1].

This study considers a problem of finding an optimal approximation of conditional information measures, like the conditional entropy, of a Bernoulli random variable (RV) given an auxiliary RV. We define an optimality of the problem in terms of the data-processing lemma of conditional information measures; and the problem aims to create an optimal auxiliary RV through partial orders of auxiliary RVs for a given RV. We describe an optimal upgradation algorithm by dynamic programming. Proving a Monge property of the dynamic programming, we describe speeding-up method of the algorithm by applying SMAWK algorithm.

This study examines multilevel channel polarization for a certain class of erasure channels with arbitrary input alphabet size. We derive limiting proportions of partially noiseless channels for such a class. One of the results of this study are proved by an argument of convergent sequences, inspired by Alsan and Telatar's simple proof of polarization [IEEE Transactions on Information Theory, vol. 62, no. 9, pp. 4873-4878, 2016], and without martingale convergence theorems for polarization process. Technical parts of this study can be found in the arXiv at [https://arxiv.org/abs/1801.04422].

This study investigates generalized Fano-type inequalities in the following senses: (i) the alphabet X of a random variable X is countably infinite; (ii) instead of a fixed finite cardinality of X, a fixed X-marginal distribution is given; (iii) information measures are generalized from the conditional Shannon entropy H(X | Y) to a general type of conditional information measures hφ(X | Y) without explicit form; and (iv) the average probability of error is defined on list-decoding rules. As a result, we give tight upper bounds on such generalized conditional information measures for a fixed X-marginal, a fixed list size, and a fixed tolerated probability of error. Then, we also clarify a sufficient condition, which the Fano-type inequalities are sharp, on the cardinality of the alphabet Y of a side information Y. Resulting Fano-type inequalities can apply to not only the conditional Shannon entropy but also the Arimoto's and Hayashi's conditional Rényi entropies. All of the proofs in this study can be found in arXiv:1801.02876 [19].

This paper examines the exact ranges between the symmetric capacity and Gallager's reliability function E0for ternary-input discrete memoryless channels (T-DMCs) under a uniform input distribution. We first derive the two extremal ternary-input strongly symmetric channels taking the maximum and minimum values of the E0function among all ternary-input strongly symmetric channels with a fixed capacity. Extending the results of ternary-input strongly symmetric channels, we second derive the exact ranges between capacity and the E0function for ternary-input Gallager-symmetric channels. We third show that the exact ranges between the symmetric capacity and the E0function of T-DMCs coincide with the ranges of ternaryinput Gallager-symmetric channels. In particular, we identify the extremal channels taking the maximum and minimum of E0among all T-DMCs with a fixed symmetric capacity. As applications of the results, we describe some bounds of error exponents for T-DMCs with a fixed symmetric capacity.

This study examines quantization for outputs of binary-input discrete memoryless channels (B-DMCs) by concatenating its output with another DMC, so-called a quantizer. As an objective function of channel quantization, we employ the α-mutual information of a B-DMC, which connects to more powerful coding theorem than the ordinary mutual information. Showing a Monge property of the α-mutual information, we propose an optimal quantizer design algorithm for given B-DMC in polynomial time complexity with respect to the output alphabet size and the quantized level. Since the proposed method employs the SMAWK algorithm due to the Monge property, our algorithm is faster than a naive dynamic programming.

This study examines sharp bounds on Arimoto's conditional Rényi entropy of order β with a fixed another one of distinct order α ≠ β. Arimoto inspired the relation between the Rényi entropy and the ℓr-norm of probability distributions, and he introduced a conditional version of the Rényi entropy. From this perspective, we analyze the ℓr-norms of particular distributions. As results, we identify specific probability distributions which achieve our sharp bounds on the conditional Rényi entropy. The sharp bounds derived in this study can be applicable to other information measures which are strictly monotone functions of the conditional Rényi entropy.

The paper examines relationships between the conditional Shannon entropy and the expectation of ℓα-norm for joint probability distributions. More precisely, we investigate the sharp bounds of the expectation of ℓα-norm with a fixed conditional Shannon entropy, and vice versa. As applications of the results, we derive the sharp bounds between the conditional Shannon entropy and several information measures which are determined by the expectation of ℓα-norm, e.g., Arimoto's conditional Rényi entropy and the conditional R-norm information. Moreover, we apply these results to discrete memoryless channels under a uniform input distribution. Then, sharp bounds are obtained for Gallager's reliability functions E0with a fixed mutual information under a uniform input distribution.

The polar transform of a binary erasure channel (BEC) can be exactly approximated by other BECs. Arikan proposed that polar codes for a BEC can be efficiently constructed by using its useful property. This study proposes a new class of arbitrary input generalized erasure channels, which can be exactly approximated the polar transform by other same channel models, as with the BEC. One of the main results is the recursive formulas of the polar transform of the proposed channel. In the study, we evaluate the polar transform by using the α-mutual information. Particularly, when the input alphabet size is a prime power, we examines the following: (i) inequalities for the average of the α-mutual information of the proposed channel after the one-step polar transform, and (ii) the exact proportion of polarizations of the α-mutual information of proposed channels in infinite number of polar transforms.

In the refinement of a channel coding theorem, error exponents characterize the exponential convergence rates of decoding error probabilities. Error exponents are sometime called as reliability functions. In this study, we consider analyzing the reliability functions based on Gallager's E0function. The region of the E0function of binary-input memoryless and symmetric channels for a fixed capacity was clarified by Guillén i Fàbregas et al. in their 2013 study. More precisely, binary erasure and binary symmetric channels have maximal and minimal E0functions, respectively, among the binary-input memoryless and symmetric channels for a fixed capacity. In this study, we extend their results from binary- to ternary-input channels that are not necessarily symmetric. First, we identify the extreme channels among the ternary-input strongly symmetric channels. Next, we identify the extreme channels among the ternary-input memoryless and symmetric channels. In addition, using channel symmetrization, we investigate whether the feasible regions of symmetric capacity and the E0functions for discrete memoryless channels (DMCs) are identical to those for symmetric channels if the channel inputs follow a uniform distribution. We describe the feasible regions for ternary-input DMCs under a uniform input distribution. In particular, we reveal the channels with maximal E0function among ternary-input DMCs for a fixed symmetric capacity and uniform input distribution.