In the last decade, the approximate vanishing ideal and its basis construction algorithms have been extensively studied in computer algebra and machine learning as a general model to reconstruct the algebraic variety on which noisy data approximately lie. In particular, the basis construction algorithms developed in machine learning are widely used in applications across many fields because of their monomial-order-free property; however, they lose many of the theoretical properties of computer-algebraic algorithms. In this paper, we propose general methods that equip monomial-order-free algorithms with several advantageous theoretical properties. Specifically, we exploit the gradient to (i) sidestep the spurious vanishing problem in polynomial time to remove symbolically trivial redundant bases, (ii) achieve consistent output with respect to the translation and scaling of input, and (iii) remove nontrivially redundant bases. The proposed methods work in a fully numerical manner, whereas existing algorithms require the awkward monomial order or exponentially costly (and mostly symbolic) computation to realize properties (i) and (iii). To our knowledge, property (ii) has not been achieved by any existing basis construction algorithm of the approximate vanishing ideal.

Ranking the set of search results according to their relevance to a user query is an important task in an Information Retrieval (IR) systems such as a Web Search Engine. Learning the optimal ranking function for this task is a challenging problem because one must consider complex non-linear interactions between numerous factors such as the novelty, authority, contextual similarity, etc. of thousands of documents that contain the user query. We model this task as a non-linear ranking problem, for which we propose Rank-PMBGP, an efficient algorithm to learn an optimal non-linear ranking function using Probabilistic Model Building Genetic Programming. We evaluate the proposed method using the LETOR dataset, a standard benchmark dataset for training and evaluating ranking functions for IR. In our experiments, the proposed method obtains a Mean Average Precision (MAP) score of 0.291, thereby significantly outperforming a non-linear baseline approach that uses Genetic Programming.

Estimation of distribution algorithms (EDAs) which deal with tree structures as GP are called as probabilistic model building GPs (PMBGPs), and they show better search performance than GP in many problems. A problem of prototype tree-based method, a type of PMBGPs, is that samplings do not always generate the most probable solution, which is the individual with the highest probability and reflects a learned distribution most. This problem wastes a part of learning and increases the number of evaluations to get an optimum solution. In order to overcome this difficulty, this paper proposes a hybrid approach using Belief propagation (BP) in sampling process. BP is an inference algorithm on graphical models and can generate the most probable solution. By applying our approach to benchmark tests, we show that the proposed method is more effective than PLS alone.

In recent years, program evolution algorithms based on the estimation of distribution algorithm (EDA) have been proposed to improve search ability of genetic programming (GP) and to overcome GP-hard problems. One such method is the probabilistic prototype tree (PPT) based algorithm. The PPT based method explores the optimal tree structure by using the full tree whose number of child nodes is maximum among possible trees. This algorithm, however, suffers from problems arising from function nodes having different number of child nodes. These function nodes cause intron nodes, which do not affect the fitness function. Moreover, the function nodes having many child nodes increase the search space and the number of samples necessary for properly constructing the probabilistic model. In order to solve this problem, we propose binary encoding for PPT. Here, we convert each function node to a subtree of binary nodes where the converted tree is correct in grammar. Our method reduces ineffectual search space, and the binary encoded tree is able to express the same tree structures as the original method. The effectiveness of the proposed method is demonstrated through the use of two computational experiments.

Estimation of distribution algorithms are evolutionary algorithms using probabilistic techniques instead of traditional genetic operators. Recently, the application of probabilistic techniques to program and function evolution has received increasing attention, and this approach promises to provide a strong alternative to the traditional genetic programming techniques. Although a probabilistic context-free grammar (PCFG) is a widely used model for probabilistic program evolution, a conventional PCFG is not suitable for estimating interactions among nodes because of the context freedom assumption. In this paper, we have proposed a new evolutionary algorithm named programming with annotated grammar estimation based on a PCFG with latent annotations, which allows this context freedom assumption to be weakened. By applying the proposed algorithm to several computational problems, it is demonstrated that our approach is markedly more effective at estimating building blocks than prior approaches.

Genetic programming (GP) is a powerful optimization algorithm that has been applied to a variety of problems. This algorithm can, however, suffer from problems arising from the fact that a crossover, which is a main genetic operator in GP, randomly selects crossover points, and so building blocks may be destroyed by the action of this operator. In recent years, evolutionary algorithms based on probabilistic techniques have been proposed in order to overcome this problem. In the present study, we propose a new program evolution algorithm employing a Bayesian network for generating new individuals. It employs a special chromosome called theexpandedparsetree,which significantly reduces the size of the conditional probability table (CPT). Prior prototype tree-based approaches have been faced with the problem of huge CPTs, which not only require significant memory resources, but also many samples in order to construct the Bayesian network. By applying the present approach to three distinct computational experiments, the effectiveness of this new approach for dealing with deceptive problems is demonstrated.

Genetic Programming (GP) which mimics the natural evolution to optimize functions and programs, has been applied to many problems. In recent years, evolutionary algorithms are seen from the viewpoint of the estimation of distribution. Many algorithms called EDAs (Estimation of Distribution Algorithms) based on probabilistic techniques have been proposed. Although probabilistic context free grammar (PCFG) is often used for the function and program evolution, it assumes the independence among the production rules. With this simple PCFG, it is not able to induce the building-blocks from promising solutions. We have proposed a new function evolution algorithm based on PCFG using latent annotations which weaken the independence assumption. Computational experiments on two subjects (the royal tree problem and the DMAX problem) demonstrate that our new approach is highly effective compared to prior approaches.

Genetic programming (GP) is a powerful optimization algorithm and has been applied to many problems. GP is an extension of genetic algorithm (GA) which can handle programs, functions, etc. GP evolves with genetic operators such as crossover and mutation. The crossover operator in GP however selects sub-trees randomly and this selection is done regardless of the problem. This gives rise to the destruction of good building blocks. Recently, probabilistic model building techniques have been applied to GP to estimate the building blocks properly. This type of algorithm is called probabilistic model building GP (PMBGP). Because GP uses many types of nodes, prior PMBGPs have been faced with the problem of huge CPT (Conditional Probability Table) size. The large CPT not only consumes a lot of memory but also requires many samples to construct networks. We propose a new PMBGP that uses Bayesian network for generating new individuals. In our approach, a special chromosome called expanded parse tree is used to improve the problem of huge CPT size.

Recently, genetic programming (GP) has been applied to the classification of gene expression data. In its typical implementation, using training data, a single rule or a single set of rules is evolved with GP, and then it is applied to test data to get generalized test accuracy. However, in most cases, the generalized test accuracy is not higher. In this paper, we propose a majority voting technique for prediction of the labels of test samples. Instead of a single rule or a single set of rules, we evolve multiple rules with GP and then apply those rules to test samples to determine their labels by using the majority voting technique. We demonstrate the effectiveness of our proposed method by performing different types of experiments on two microarray data sets.