Decision tree algorithms are one of the most popular methods in machine learning. However, most decision tree algorithms do not assume a stochastic model behind data. On the other hand, a meta-tree was recently proposed as a stochastic model with a tree structure. The prediction under the assumption of the meta-tree is decided using Bayesian decision theory. Although the optimal prediction can be calculated with an assumption of a known meta-tree, an approximation is necessary to obtain a prediction under the problem setting of an unknown meta-tree because of the marginalization of all possible meta-trees. In this paper, we propose an approximation method, where a subset of meta-trees is sequentially constructed, and the prediction is made by weighting the meta-trees. For the approximation, we clarify the approaches of constructing the subset and making predictions with weighted meta-trees. We also examine the effectiveness of the approaches in an experiment using synthetic data. In addition, we conduct an experiment on benchmark data to confirm the performance of the proposal.

This study aims to identify rectangular regions where the features follow an independent and identically distributed probability. To achieve this, we employ a method that divides large regions into smaller ones. We model the image using a quadtree structure and assume prior distributions for the image, labels, and quadtree. Using Bayesian decision theory, we derive the optimal estimation solution. Few studies have considered prior distributions on images for optimal estimation. We reduced computational complexity by selecting an appropriate prior distribution for the quadtree. Additionally, we estimated the parameters of each probability distribution using the training data.

Predictions using a combination of decision trees are known to be effective in machine learning. Typical ideas for constructing a combination of decision trees for prediction are bagging and boosting. Bagging independently constructs decision trees without evaluating their combination performance and averages them afterward. Boosting constructs decision trees sequentially, only evaluating a combination performance of a new decision tree and the fixed past decision trees at each step. Therefore, neither method directly constructs nor evaluates a combination of decision trees for the final prediction. When the final prediction is based on a combination of decision trees, it is natural to evaluate the appropriateness of the combination when constructing them. In this paper, we propose a new algorithmic framework that constructs decision trees simultaneously and evaluates their combination performance throughout the construction process. Our framework repeats two procedures. In the first procedure, we construct new candidates of combinations of decision trees to find a proper combination of decision trees. In the second procedure, we evaluate each combination performance of decision trees under some criteria and select a better combination. To confirm the performance of the proposed framework, we experiment with synthetic and benchmark data.

In recent years, Bayesian counterparts of the context tree weighting method are studied for many tasks. All these tasks require a hyperparameter setting of the prior distribution for context tree models. Therefore, we provide a framework for statistically learning these hyperparameters from data. Specifically, we consider a hierarchical Bayesian model that assumes hyperprior distributions behind the hyperparameters and learn them using an empirical variational Bayesian (EVB) method. This is the first study to propose an EVB method on the Bayesian context trees. The derived algorithm has a suggestive form that consists of subroutines partially optimal to each local probabilistic model.

This paper presents a weather map prediction method using RGB metaphorical feature extraction for atmospheric pressure patterns. In the field of meteorological science, predicting weather based on the analysis of observational data and the knowledge of weather experts is crucial. Weather experts draw weather maps based on air pressure distribution; hence, we believe that weather maps entail the interpretations of weather experts. In this study, we improved the prediction accuracy by using machine learning to recognize patterns of qualitative expert interpretations that cannot be predicted by analyzing observed data alone. The proposed method can be realized via two steps. The first is developing a module for extracting pressure pattern features from a weather map. Certain features, such as tropical cyclones or atmospheric high/low pressure distributions, are emphasized in weather maps to facilitate better understanding of the weather features. Therefore, we can predict weather features based on the knowledge of weather experts using data that contain their interpretations, particularly weather maps. The developed module extracts the atmospheric pressure features from the current weather map as an RGB metaphorical gradation map. The second step is developing a module to design a predicted weather map using the extracted features. The weather map of the following day is predicted using pix2pix. To the best of our knowledge, our method for extracting features from weather maps is the first to create a predicted weather map automatically.

The context tree source is a source model in which the occurrence probability of symbols is determined from a finite past sequence, and is a broader class of sources that includes i.i.d. and Markov sources. This paper proposes a source model such that its subsequence is generated from a different context tree model. The Bayes code for such sources requires weighting of the posterior probability distributions for the change patterns of the context tree source and all possible context tree models. Therefore, the challenge is how to reduce this exponential order computational complexity. In this paper, we assume a special class of prior probability distribution of change patterns and context tree models, and propose an efficient Bayes coding algorithm whose computational complexity is the polynomial order. A full version of this paper is accessible at: https://arxiv.org/abs/2105.05163