This paper proposes the soft Bayesian context tree model (Soft-BCT), which is a novel BCT model for real-valued time series. The Soft-BCT considers soft (probabilistic) splits of the context space, instead of hard (deterministic) splits of the context space as in the previous BCT for real-valued time series. A learning algorithm of the Soft-BCT is proposed based on the variational inference. The results of experiments demonstrate the superiority of the Soft-BCT compared to the previous BCT for some datasets.

Deterministic decision trees have difficulty in evaluating uncertainty especially for small samples. To solve this problem, we interpret the decision trees as stochastic models and consider prediction problems in the framework of Bayesian decision theory. Our models have three kinds of parameters: a tree shape, leaf parameters, and inner parameters. To make Bayesian optimal decisions, we have to calculate the posterior distribution of these parameters. Previously, two types of methods have been proposed. One marginalizes out the leaf parameters and samples the tree shape and the inner parameters by Metropolis-Hastings (MH) algorithms. The other marginalizes out both the leaf parameters and the tree shape based on a concept called meta-trees and approximates the posterior distribution for the inner parameters by a bagging-like method. In this paper, we propose a novel MH algorithm where the leaf parameters and the tree shape are marginalized out by using the meta-trees and only the inner parameters are sampled. Moreover, we update all the inner parameters simultaneously in each MH step. This algorithm accelerates the convergence and mixing of the Markov chain. We evaluate our algorithm on various benchmark datasets with other state-of-the-art methods. Further, our model provides a novel statistical evaluation of feature importance.

One of the major models in causal inference is the Rubin Causal Model. In this model, treatment effect estimation is considered within the framework of missing data analysis. However, many previous studies based on this model have not adopted the framework of missing data analysis and instead have moved to discussions based on conditional probability distributions for outcome variables. In this study, we propose a framework for causal inference that does not rely on missing data analysis, using a generalized probabilistic model for treatment effect estimation and statistical decision theory. Furthermore, to verify the effectiveness of this framework, we analytically derive the optimal treatment effect estimation under the Bayesian criterion based on Bayesian decision theory by assuming a linear basis function model. In addition, we compare the proposed method for deriving the optimal estimation with conventional treatment effect estimation methods through simulations. This comparison clarifies the properties of the conventional methods.

Decision tree algorithms are one of the most popular methods in machine learning. However, most decision tree algorithms do not assume a stochastic model behind data. On the other hand, a meta-tree was recently proposed as a stochastic model with a tree structure. The prediction under the assumption of the meta-tree is decided using Bayesian decision theory. Although the optimal prediction can be calculated with an assumption of a known meta-tree, an approximation is necessary to obtain a prediction under the problem setting of an unknown meta-tree because of the marginalization of all possible meta-trees. In this paper, we propose an approximation method, where a subset of meta-trees is sequentially constructed, and the prediction is made by weighting the meta-trees. For the approximation, we clarify the approaches of constructing the subset and making predictions with weighted meta-trees. We also examine the effectiveness of the approaches in an experiment using synthetic data. In addition, we conduct an experiment on benchmark data to confirm the performance of the proposal.

Private Information Retrieval (PIR) is a mechanism for efficiently downloading messages while keeping the index secret. The information-theoretic upper bound on efficiency has been proved in previous studies; PIR properties and the proofs of capacity were notated in terms of entropy and probability. However, in order to construct a linear PIR, it is necessary to clarify the properties of the query matrix. In this study, we prove the necessary and sufficient conditions for PIR properties, and represent them in matrix form. We also show a PIR construction method that satisfies the conditions.

This study aims to identify rectangular regions where the features follow an independent and identically distributed probability. To achieve this, we employ a method that divides large regions into smaller ones. We model the image using a quadtree structure and assume prior distributions for the image, labels, and quadtree. Using Bayesian decision theory, we derive the optimal estimation solution. Few studies have considered prior distributions on images for optimal estimation. We reduced computational complexity by selecting an appropriate prior distribution for the quadtree. Additionally, we estimated the parameters of each probability distribution using the training data.

Predictions using a combination of decision trees are known to be effective in machine learning. Typical ideas for constructing a combination of decision trees for prediction are bagging and boosting. Bagging independently constructs decision trees without evaluating their combination performance and averages them afterward. Boosting constructs decision trees sequentially, only evaluating a combination performance of a new decision tree and the fixed past decision trees at each step. Therefore, neither method directly constructs nor evaluates a combination of decision trees for the final prediction. When the final prediction is based on a combination of decision trees, it is natural to evaluate the appropriateness of the combination when constructing them. In this paper, we propose a new algorithmic framework that constructs decision trees simultaneously and evaluates their combination performance throughout the construction process. Our framework repeats two procedures. In the first procedure, we construct new candidates of combinations of decision trees to find a proper combination of decision trees. In the second procedure, we evaluate each combination performance of decision trees under some criteria and select a better combination. To confirm the performance of the proposed framework, we experiment with synthetic and benchmark data.

In recent years, Bayesian counterparts of the context tree weighting method are studied for many tasks. All these tasks require a hyperparameter setting of the prior distribution for context tree models. Therefore, we provide a framework for statistically learning these hyperparameters from data. Specifically, we consider a hierarchical Bayesian model that assumes hyperprior distributions behind the hyperparameters and learn them using an empirical variational Bayesian (EVB) method. This is the first study to propose an EVB method on the Bayesian context trees. The derived algorithm has a suggestive form that consists of subroutines partially optimal to each local probabilistic model.

This paper investigates the problem of variable-length source coding with the criteria of the normalized cumulant generating function of codeword lengths and the excess distortion probability. We analyze the non-asymptotic fundamental limit of the normalized cumulant generating function of codeword lengths under the constraint that the excess distortion probability is allowed up to$\epsilon \in [0,1)$. Our non-asymptotic achievability and converse bounds are characterized by the quantity related to the Rényi entropy.

Most previous studies on lossless image compression have focused on improving preprocessing functions to reduce the redundancy of pixel values in real images. However, we assumed stochastic generative models directly on pixel values and focused on achieving the theoretical limit of the assumed models. In this study, we proposed a stochastic model based on improper quadtrees. We theoretically derive the optimal code for the proposed model under the Bayes criterion. In general, Bayes-optimal codes require an exponential order of calculation with respect to the data lengths. However, we propose an efficient algorithm that takes a polynomial order of calculation without losing optimality by assuming a novel prior distribution.

The Stratonovich’s value of information (VoI) is quantity that measures how much inferential gain is obtained from noisy data under information leakage constraint. In this paper, we introduce a generalized VoI for a general loss function and general information leakage. Then we derive an upper bound of the generalized VoI. Moreover, for a classical loss function, we provide a achievable condition of the upper bound which is weaker than that of in previous studies. Since VoI can be viewed as a formulation of a privacy-utility trade-off (PUT) problem, we provide an interpretation of the achievable condition in the PUT context.

The hierarchical and recursive expressive capability of rooted trees is applicable to represent statistical models in various areas, such as data compression, image processing, and machine learning. On the other hand, such hierarchical expressive capability causes a problem in tree selection to avoid overfitting. One unified approach to solve this is a Bayesian approach, on which the rooted tree is regarded as a random variable and a direct loss function can be assumed on the selected model or the predicted value for a new data point. However, all the previous studies on this approach are based on the probability distribution on full trees, to the best of our knowledge. In this paper, we propose a generalized probability distribution for any rooted trees in which only the maximum number of child nodes and the maximum depth are fixed. Furthermore, we derive recursive methods to evaluate the characteristics of the probability distribution without any approximations.

Suppose that we have two training sequences generated by parametrized distributions$P_{\theta}$and$P_{\varepsilon^{*}}$, where$\theta$* and$\xi^{*}$are unknown true parameters. Given training sequences, we study the problem of classifying whether a test sequence was generated according to$P_{\theta}$* or$P_{\xi^{*}}$. This problem can be thought of as a hypothesis testing problem and our aim is to analyze the weighted sum of type-I and type-II error probabilities. Utilizing the analysis of the codeword lengths of the Bayes code, our previous study derived more refined bounds on the error probability than known previously. However, our previous study had the following deficiencies: i) the prior distributions of$\theta$and$\xi$are the same; ii) the prior distributions of two hypotheses are uniform; iii) no numerical calculation at finite blocklength. This study solves these problems. We remove the restrictions i) and ii) and derive more general results than obtained previously. To deal with problem iii), we perform a numerical calculation for a concrete model.

We consider the problem of publishing data with utility and privacy guarantees in a statistical decision-theoretical framework. In this framework, we introduce a statistical decision-theoretic quantity called average gain for measuring not only privacy but also utility. We also show a relationship between the average gain and the $\alpha$-leakage, a tunable leakage measure proposed by Liao et at. Moreover, we formulate the privacyutility trade-off (PUT) problem using Stratonovich’s value of information (VoI) and present an analysis of the PUT.

Explicit assumption of stochastic data generative models is a remarkable feature of lossless compression of general data in information theory. However, current lossless image coding mostly focus on coding procedures without explicit assumption of the stochastic generative model. Therefore, we have difficulty discussing the theoretical optimality of the coding procedure to the stochastic generative model. In this paper, we solve this difficulty by constructing a stochastic generative model by interpreting the previous coding procedure from another perspective. An important problem of our approach is how to learn the hyperparameters of the stochastic generative model because the optimality of our coding algorithm is guaranteed only asymptotically and the hyperparameter setting still affects the expected code length for finite length data. For this problem, we use Bayesian hierarchical modeling and confirm its effect by numerical experiments. In lossless image coding, this is the first study assuming such an explicit stochastic generative model and learning its hyperparameters, to the best of our knowledge.

The context tree source is a source model in which the occurrence probability of symbols is determined from a finite past sequence, and is a broader class of sources that includes i.i.d. and Markov sources. This paper proposes a source model such that its subsequence is generated from a different context tree model. The Bayes code for such sources requires weighting of the posterior probability distributions for the change patterns of the context tree source and all possible context tree models. Therefore, the challenge is how to reduce this exponential order computational complexity. In this paper, we assume a special class of prior probability distribution of change patterns and context tree models, and propose an efficient Bayes coding algorithm whose computational complexity is the polynomial order. A full version of this paper is accessible at: https://arxiv.org/abs/2105.05163

In this paper, we propose a novel stochastic model based on the quadtree, so that our model effectively represents the variable block size segmentation of images. Then, we construct the Bayes code for the proposed stochastic model. In general, the computational cost to calculate the posterior distribution required in the Bayes code increases exponentially with respect to the data size. However, we introduce an efficient algorithm to calculate it in the polynomial order of the data size without loss of the optimality. Some experiments are performed to confirm the flexibility of the proposed stochastic model and the efficiency of the introduced algorithm.

We propose two new criteria to understand the advantage of deepening neural networks. It is important to know the expressivity of functions computable by deep neural networks in order to understand the advantage of deepening neural networks. Unless deep neural networks have enough expressivity, they cannot have good performance even though learning is successful. In this situation, the proposed criteria contribute to understanding the advantage of deepening neural networks since they can evaluate the expressivity independently from the efficiency of learning. The first criterion shows the approximation accuracy of deep neural networks to the target function. This criterion has the background that the goal of deep learning is approximating the target function by deep neural networks. The second criterion shows the property of linear regions of functions computable by deep neural networks. This criterion has the background that deep neural networks whose activation functions are piecewise linear are also piecewise linear. Furthermore, by the two criteria, we show that to increase layers is more effective than to increase units at each layer on improving the expressivity of deep neural networks.

Suppose that we have two training sequences generated by parametrized distributions ${P_{\theta _1^{\ast}}}$ and ${P_{\theta _2^{\ast}}}$, where $\theta _1^{\ast}$ and $\theta _2^{\ast}$ are unknown. Given training sequences, we study the problem of classifying whether a test sequence was generated according to ${P_{\theta _1^{\ast}}}$ or ${P_{\theta _2^{\ast}}}$. This problem can be thought of as a hypothesis testing problem and the weighted sum of type-I and type-II error probabilities is analyzed. To prove the results, we utilize the analysis of the codeword lengths of the Bayes code. It is shown that upper and lower bounds of the probability of error are characterized by the terms containing the Chernoff information, the dimension of a parameter space, and the ratio of the length between the training sequences and the test sequence. Further, we generalize the part of the preceding results to multiple hypotheses setup.

We consider a distributed learning problem in which the computation is carried out on a system consisting of a master node and multiple worker nodes. In such systems, the existence of slow-running machines called stragglers will cause a significant decrease in performance. Recently, coding theoretic framework, which is named Gradient Coding (GC), for mitigating stragglers in distributed learning has been established by Tandon et al. Most studies on GC are aiming at recovering the gradient information completely assuming that the Gradient Descent (GD) algorithm is used as a learning algorithm. On the other hand, if the Stochastic Gradient Descent (SGD) algorithm is used, it is not necessary to completely recover the gradient information, and its unbiased estimator is sufficient for the learning. In this paper, we propose a distributed SGD scheme using Low Density Generator Matrix (LDGM) codes. In the proposed system, it may take longer time than existing GC methods to recover the gradient information completely, however, it enables the master node to obtain a high-quality unbiased estimator of the gradient at low computational cost and it leads to overall performance improvement.

This paper investigates the problem of guessing subject to distortion, which was introduced by Arikan and Merhav. While the primary concern of the previous study was asymptotic analysis, our primary concern is non-asymptotic analysis. We prove non-asymptotic achievability and converse bounds of the moment of the number of guesses without side information (resp. with side information) by using a quantity based on the Rényi entropy (resp. the Arimoto-Rényi conditional entropy). Also, we introduce an error probability and show similar results. Further, from our bounds, we derive a single-letter characterization of the asymptotic exponent of guessing moment for a stationary memoryless source.

A spatially “Mt. Fuji” coupled low-density parity check (LDPC) ensemble is a modified version of the original spatially coupled (SC) LDPC ensemble. Its desirable properties are first observed in experimentally. The decoding error probability in the error floor region over the binary erasure channel (BEC) is theoretically analyzed later. In this paper, as the last piece of the theoretical analysis over the BEC, we analyze the decoding error probability in the waterfall region by modifying the covariance evolution which has been used to analyze the original SC-LDPC ensemble.

We consider the model selection of the hierarchical mixture of experts (HME). The HME is a tree-structured probabilistic model for regression and classification. The HME model has high prediction accuracy and high interpretability, however, the estimation of the parameters tends to overfit due to the complexity of the model. In order to mitigate the overfitting problem, in previous studies, several Bayesian estimation methods for the HME parameters have been proposed. In these studies, the true model that generates data is fixed. In general, however, the true model is unknown. Model selection is one of the most important and difficult problems of regression and classification. For the Bayesian HME, the model is determined by the tree structure, the form of the prior distribution and its parameters, however, only the tree structure is considered as a model parameter in previous studies. In this paper, we consider all of these as model parameters and extend the model selection method. Then, we propose a maximum a posteriori (MAP) estimation method of the Bayesian HME model selection. The approximate posterior probability of each model is calculated by the variational lower bound. We show the effectiveness of the proposed method by numerical experiments and discuss the results applied to actual data sets.

In this paper, we propose a new distributed optimization method that computes a Lasso estimator for logistic regression in the case when two parties have explanatory variables corresponding to distinct attributes. An existing protocol using the alternating direction method of multipliers (ADMM) for linear regression can be applied to logistic regression. However, this protocol needs an underlying iterative method such as the gradient method. We show that the proposed protocol using the generalized Bregman ADMM, which removes the necessity to use the underlying iterative method, requires lower computational and communication complexity.

This paper considers the problem of variable-length lossy source coding. The performance criteria are the excess distortion probability and the cumulant generating function of codeword lengths. We derive a non-asymptotic fundamental limit of the cumulant generating function of codeword lengths allowing positive excess distortion probability. It is shown that the achievability and converse bounds are characterized by the Rényi entropy-based quantity. In the proof of the achievability result, the explicit code construction is provided. Further, we investigate an asymptotic single-letter characterization of the fundamental limit for a stationary memoryless source. A full version of this paper is accessible at: http://arxiv.org/abs/1801.02496

The hierarchical mixture of experts (HME) is a tree-structured probabilistic model for regression and classification. The HME has a considerable expression capability, however, the estimation of the parameters tends to overfit due to the complexity of the model. To avoid this problem, regularization techniques are widely used. In particular, it is known that a sparse solution can be obtained by L1 regularization. From a Bayesian point of view, regularization techniques are equivalent to assume that the parameters follow prior distributions and find the maximum a posteriori probability estimator. It is known that L1 regularization is equivalent to assuming Laplace distributions as prior distributions. However, it is difficult to compute the posterior distribution if Laplace distributions are assumed. In this paper, we assume that the parameters of the HME follow hierarchical prior distributions which are equivalent to Laplace distribution to promote sparse solutions. We propose a Bayesian estimation algorithm based on the variational method. Finally, the proposed algorithm is evaluated by computer simulations.

This paper shows some new results for the problem of variable-length lossy source coding. We deal with the case where both the excess distortion probability and the overflow probability of codeword lengths are less than or equal to positive constants. Our previous study for the problem of variable-length (noiseless) lossy source coding has derived the general formula of the infimum of the thresholds on the overflow probability by using the quantity based on the smooth max entropy. This study extends this result in two directions. First, we derive the single-letter characterization of the infimum of the thresholds on the overflow probability for stationary memoryless sources. Second, for the problem of variable-length noisy lossy source coding, also known as the problem of remote lossy source coding, we establish the general nonasymptotic formula on the converse bound by using the new quantity based on the smooth max entropy.

This paper considers the problem of variable-length intrinsic randomness. We propose the average variational distance as the performance criterion from the viewpoint of a dual relationship with the problem formulation of variable-length resolvability. Previous study has derived the general formula of the ϵ-variable-length resolvability. We derive the general formula of the ϵ-variable-length intrinsic randomness. Namely, we characterize the supremum of the mean length under the constraint that the value of the average variational distance is smaller than or equal to a constant ϵ. Our result clarifies a dual relationship between the general formula of ϵ-variable-length resolvability and that of ϵ-variable-length intrinsic randomness. We also derive a lower bound of the quantity characterizing our general formula.

In coding theory, it is important to find upper bounds for the code size given a code length and minimum distance. The Hamming bounds and Linear Programming (LP) bounds were proposed in previous works. On the other hand, Masnick et al. proposed Unequal Error Protection (UEP) codes and modified Hamming bounds as upper bounds for the code size of UEP codes. In our previous work, we defined 2-level UEP codes as a subclass of UEP codes, and derived LP bounds for 2-level UEP codes. In this paper, we define multi-level UEP codes by extending 2-level UEP codes, and derive LP bounds for multi-level UEP codes. Moreover, we show that LP bounds for UEP codes are tighter upper bound than modified Hamming bounds.

In this manuscript, we investigate a collaborative filtering method to characterize consumption behavior of customers and services with various attributes for marketing. We assume that each customer and service have the invisible attribute which is called latent class. Assuming a combination of attribute values of a customer and service is classified to a latent class, furthermore, we propose a new Bayesian statistical model that consumption behavior is probabilistically arise based on a latent class combination of a customer, service and attribute values. Then, we show the method to estimate parameters of a statistical model based on the variational Bayes method and the mean field approximation. Consequently, we show the effectiveness of the proposed model and the estimation method by simulation.

This paper investigates the problem of variable-length lossy source coding. We deal with the case where both the excess distortion probability and the overflow probability of codeword length are less than or equal to positive constants. The infimum of the thresholds on the overflow probability is characterized by a smooth max entropy-based quantity. Both non-asymptotic and asymptotic cases are analyzed. To show the achievability results, we do not utilize the random coding argument but give an explicit code construction.

In this paper, we develop a new decoding algorithm of binary linear codes for symbol-pair read channel. The Symbol-pair read channel has recently been introduced by Cassuto and Blaum to model channel whose write resolution is higher than read resolution. The proposed decoding algorithm is based on the linear programming (LP). It is proved that the proposed LP decoder has the maximum-likelihood (ML) certificate property, i.e., the output of the decoder is guaranteed to be the ML codeword when it is integral. We also introduce the fractional pair distance dfpof the code which is a lower bound on the minimum pair distance. It is proved that the proposed LP decoder corrects up to ⌈dfp/2⌉ - 1 errors.

This paper considers universal lossless variable-length source coding problem and deals with one of the fundamental limits and pointwise asymptotics of the Bayes code for stationary ergodic finite order Markov sources. As investigation of the fundamental limits, we show upper and lower bounds of the minimum rate such that the probability which exceeds it is less than ε ∈ (0, 1). Furthermore, we prove that the codeword length of the Bayes code satisfies the asymptotic normality (pointwise √n asymptotics) and the law of the iterated logarithm (pointwise √n log log n asymptotics), where n represents length of a source sequence and “log” is the natural logarithm.

In this paper, we consider a completion problem of multiple related matrices. Matrix completion problem is the problem to estimate unobserved elements of the matrix from observed elements. It has many applications such as collaborative filtering, computer vision, biology, and so on. In cases where we can obtain some related matrices, we can expect that their simultaneous completion has better performance than completing each matrix independently. Collective matrix factorization is a powerful approach to jointly factorize multiple matrices. However, existing completion algorithms for the collective matrix factorization have some drawbacks. One is that most existing algorithms are based on non-convex formulations of the problem. Another is that only a few existing algorithms consider the strength of the relation among matrices and it results in worse performance when some matrices are actually not related. In this paper, we formulate the multiple matrix completion problem as the convex optimization problem. Moreover, it considers the strength of the relation among matrices. We also develop an optimization algorithm which solves the proposed problem efficiently based on the alternating direction method of multipliers (ADMM). We verify the effectiveness of our approach through numerical experiments on both synthetic data and real data set: MovieLens.

F. P. Preparata et al. proposed a fault diagnosis model (PMC model) to find all fault units in the multicomputer system by using outcomes that each unit tests some other units. T. Kohda proposed a highly structured(HS) system and the syndrome analysis method(SAM) to diagnose from local testing results. In this paper, we introduce the maximum a posteriori probability algorithm(MAPDA) for the HS system in the probabilistic fault model. Analyzing the MAPDA, we show that the SAM is closer to the MAPDA as the fault probability becomes smaller. Finally, we show the robustness of the SAM in the HS system.

Lossless variable-length source coding with unequal cost function is considered for general sources. In this problem, the codeword cost instead of codeword length is important. The infimum of average codeword cost has already been determined for general sources. We consider the overflow probability of codeword cost and determine the infimum of achievable overflow threshold. Our analysis is on the basis of information-spectrum methods and hence valid through the general source.

Experiments usually aim to study how changes in various factors affect the response variable of interest. Since the model used most often at present in experimental design is expressed through the effect of each factor, it is easy to understand how each factor affects the response variable. However, since the model contains redundant parameters, a considerable amount of time is often necessary to implement the procedure for estimating the effects. On the other hand, it has recently been shown that the model in experimental design can also be expressed in terms of an orthonormal system. In this case, the model contains no redundant parameters. Moreover, the theorem with respect to the sum of squares for the 2-factor interaction, needed in the analysis of variance (ANOVA) has been obtained. However, 3-factor interaction is often to be considered in real cases, but the theorem with respect to the sum of squares for the 3-factor interaction has not been obtained up to now. In this paper, we present the theorem with respect to the sum of squares for the 3-factor interaction in a model based on an orthonormal system. Furthermore, we can also obtain the theorem for interactions with 4 or more factors by the similar proof. Hence, in any real case, we can execute ANOVA in the model based on an orthonormal system.

We discuss disk allocation methods for Cartesian product files by introducing error correcting codes, and have clarified the performance of the methods by system evaluation models developed by using rate distortion theory. Let us assume qnCartesian product files with n attributes and q actual values in each attribute, and store qnfiles into G(≤ qn) disks. For a partial match access request, we represent new disk allocation methods which able to access the disks in parallel as much as possible, where the partial match access request includes an indefinite case (don't care: “*”) in some attributes and the * requires to access the files with corresponding to the attribute for the all actual attribute values. In this paper, we propose to apply unequal error protection codes to the case where the probabilities of occurrence of the * in the attributes for a partial match access request are not the same. We show the disk allocation methods have desirable properties as n becomes large.

F.P.Preparata et al. have proposed a fault diagnosis model to find all faulty units in the multicomputer system by using outcomes which each unit tests some other units. In this paper, for probabilistic diagnosis models, we show an efficient diagnosis algorithm to obtain a posteriori probability that each of units is faulty given the test outcomes. Furthermore, we propose a method to analyze the diagnostic error probability of this algorithm.

Allocation methods for Cartesian product files on multiple disks using linear error-correcting codes are discussed. In this paper, we propose an allocation method using unequal error protection (UEP) codes. Codewords of an UEP code have some special bits which are protected against a greater number of errors than the other bits. We firstly assume a model that “*”, which means “don't care”, appears with different probability in each attribute of queries. In this case, the average access time can be calculated using the split distance distribution. Finally, we illustrate the average access time of the methods using UEP codes.

Experiments usually aim to study how changes in various factors affect the response variable of interest. Since the response model used most often at present in experimental design is expressed through the effect of each factor, it is straightforward to ascertain how each factor affects the response variable. However, since the response model contains redundant parameters, we must calculate the degrees of freedom defined by the number of independent parameters in the analysis of variance. In this paper, we show that through a description of experimental design based on an orthonormal system, the response model can be expressed using only independent parameters. Hence, we do not have to calculate the degrees of freedom defined by the number of independent parameters.

We consider the source coding problem with side information. Especially, we consider the FV code in the case that the encoder and the decoder can see side information. We obtain the condition that there exists a FV code under the condition that the overflow probability is smaller than or equal to some constant.

Lossless fixed-to-variable(FV) length codes are considered. The overflow probability is one of criteria that evaluate the performance of FV code. In the single source coding problem, there were many researches on the overflow probability. Recently, the source coding problem for correlated sources, such as Slepian-Wolf coding problem or source coding problem with side information, is one of main topics in information theory. In this paper, we consider the source coding problem with side information. Especially, we consider the FV code in the case that the encoder and the decoder can see side information. In this case, several codes were proposed and their mean code lengths were analyzed. However, there was no research about the overflow probability. We shall show two lemmas about the overflow probability. Then we obtain the condition that there exists a FV code under the condition that the overflow probability is smaller than or equal to some constant.

Recently, computing the capacity region of the degraded broadcast channel (DBC) was showed as a nonconvex optimization problem by Calvo et al. There seems to be no efficient method to solve in polynomial time due to the lack of convexity. In other nonconvex optimization problem, however, Kumar et al showed that Arimoto-Blahut type algorithm converges to the global optimum when some conditions hold. In this paper, we present Arimoto-Blahut type algorithm toward computing the capacity region of the DBC. By using Kumar's method, we prove the global convergence of the algorithm when some conditions hold and derive an expression for its convergence rate.

The context tree models are widely used in a lot of research fields. Patricia like trees are applied to the context trees that are expanded according to the increase of the length of a source sequence in the previous researches of non-predictive source coding and model selection. The space complexity of the Patricia like context trees are O(t) where t is the length of a source sequence. On the other hand, the predictive Bayes source coding algorithm cannot use a Patricia like context tree, because it is difficult to hold and update the posterior probability parameters on a Patricia like tree. So the space complexity of the expanded trees in the predictive Bayes coding algorithm is O(t2). In this paper, we propose an efficient predictive Bayes coding algorithm using a new representation of the posterior probability parameters and the compact context tree holding the parameters whose space complexity is O(t).

In this paper, we generalize the sampling theorem for bandlimited functions over the Boolean domain to a sampling theorem for bandlimited functions over a GF(q)ndomain. We also present a theorem for the relation between the parity check matrix of a linear code and any distinct error vectors. Lastly, we clarify the relation between the sampling theorem for functions over a GF (q)ndomain and linear codes.

This paper introduces two channel models of correlated parallel channels. Then we analyze structure of error correcting codes over these correlated parallel channels. We derive necessary and sufficient conditions for these codes and some code construction is presented. We also show some upper and lower bounds on the coding rate of the error correcting codes for correlated parallel channels. The introduced channel models are related to burst error channels and the codes analyzed in this paper can be used as asymmetric interleaving codes for burst error channels.

In this paper, transmission schemes with noiseless and high capacity feedback channel is considered. We propose two types of transmission schemes using LDPC codes and clarify the density evolution analysis method for these proposed schemes. We investigate the performance of the proposed schemes by the density evolution analysis and computer simulations. The result shows some interesting characteristics for schemes with high capacity feedback channel.

In this paper, we generalize the achievability of variable-length coding from two viewpoints. One is the definition of an overflow probability, and the other is the definition of an achievability. We define the overflow probability as the probability of codeword length, not per symbol, is larger thanetanand we introduce theisin-achievability of variable-length codes that implies an existence of a code for the source under the condition that the overflow probability is smaller than or equal toisin. Then we show that theisin-achievability of variable-length codes is essentially equivalent to theisin-achievability of fixed-length codes for general sources. Moreover we show the condition ofisin-achievability for some restricted sources givenisin.

In this paper, we present an iterative algorithm for computing the secrecy capacity of broadcast channel with confidential message (BCC) in the situation that the main channel is less noisy than the eavesdropper's channel. The global convergence of the algorithm is proved, and an expression for its convergence rate is derived.

Digital fingerprinting, a copyright protection technique for digital contents, is considered. Digital fingerprinting should deter collusion attacks, where several fingerprinted copies of the same content are mixed to disturb their fingerprints. In this paper, we consider the averaging attack, which has effect for multimedia fingerprinting. We propose new collusion-secure fingerprinting codes based on finite geometries (FGs) which increase the rate of conventional collusion-secure codes, while they guarantee to identify the same number of colluders. Due to the new FG-based fingerprinting codes, the system can deal with a larger number of users to distribute a digital content.

The problem of universal codes with side information is investigated from Bayes criterion. We propose side information context tree models which are an extension of context tree models to sources with side information. Assuming a special class of the prior distributions for side information context tree models, we propose an efficient algorithm of Bayes code for the models. The asymptotic code length of the Bayes codes with side information is also investigated

The best-known algorithm for the decoding of low-density parity-check (LDPC) codes is the sum-product algorithm (SPA). The SPA is a message-passing algorithm on a graphical model called a factor graph (FG). The performance of the SPA depends on a structure of loops in a FG. Pearl showed that loops in a graphical model could be erased by the clustering method. This method clusters plural nodes into a single node. In this paper, we show several examples about a decoding on a FG to which the clustering method is applied. And we propose an efficient decoding algorithm for it. For a binary erasure channel (BEC), the performance with this method goes up clearly.

In this paper, we discuss the difference in code lengths between the code based on the minimum description length (MDL) principle (the MDL code) and the Bayes code under the condition that the same prior distribution is assumed for both codes. It is proved that the code length of the Bayes code is smaller than that of the MDL code by o(1) or O(1) for the discrete model class and by O(1) for the parametric model class. Because we can assume the same prior for the Bayes code as for the code based on the MDL principle, it is possible to construct the Bayes code with equal or smaller code length than the code based on the MDL principle. From the viewpoint of mean code length per symbol unit (compression rate), the Bayes code is asymptotically indistinguishable from the MDL two-stage codes.

Discusses a problem of deduction with uncertainty that has been dealt with by various diagnostic expert systems. First, we propose a mathematical framework of deductive reasoning with uncertainty. The subject of the reasoning is the calculation of conditional probabilities. Second, we establish a new reasoning method. Our deduction algorithm can compute the conditional probabilities precisely. To put it another way around, the result minimizes the divergence.

Considering inductive inference and deductive inference as not individual processes but a serial process of information processing, the serial inference procedure is studied for two purposes: the compression of observed facts and the prediction of new well-formed formulas. A serial inference process scheme that uses the correspondence to source coding and prediction problems is proposed. The optimal inference procedures for the two purposes are shown in the proposed scheme.>

The problem of distortionless encoding when the parameters of the probabilistic model of a source are unknown is considered from a statistical decision theory point of view. A class of predictive and nonpredictive codes is proposed that are optimal within this framework. Specifically, it is shown that the codeword length of the proposed predictive code coincides with that of the proposed nonpredictive code for any source sequence. A bound for the redundancy for universal coding is given in terms of the supremum of the Bayes risk. If this supremum exists, then there exists a minimax code whose mean code length approaches it in the proposed class of codes, and the minimax code is given by the Bayes solution relative to the prior distribution of the source parameters that maximizes the Bayes risk.>