Solving large-scale, constrained, and nonlinear optimization problems is crucial for the Internet of Things (IoT) due to its wide range of real-life applications. However, there is no unified approach for handling constraints and optimizing objective functions. This article proposes a tri-objective general framework (TriGF) and an efficient differential evolution (DE) method enhanced with adaptive gradient-based mutation (AGM), termed AGM-DE. Within the TriGF, AGM-DE explores the entire feasible region by considering both constraints and the objective function. The goal is to achieve global optimality and fast convergence for the self-assembly of satellite IoT devices under constraints. AGM is an adaptive refinement technique that uses gradient information to reduce the search space and speed up optimization. In our AGM approach, we incorporate gradient information from the objective function to mitigate the negative effects of classic constraint-based gradient descent and reduce its inherent greediness. To validate AGM-DE’s effectiveness, we conducted extensive simulations on 57 benchmark problems with diverse dimensions and constraints. The results demonstrate AGM-DE’s exceptional ability to manage constraints in 56 of these 57 test functions, outperforming five leading methods in optimization efficacy and consistency. We also assessed AGM-DE’s application in optimizing IoT device self-assembly within a satellite layout, subject to heat pipe constraints. Comparative analyses highlight AGM-DE’s robustness and superior search capabilities in deriving layout schemes. Remarkably, these schemes outperform existing best known solutions for IoT configurations involving 40 to 90 nodes with 80 to 180 variables, confirming AGM-DE’s suitability for a wide range of large-scale constrained IoT challenges.

Recently, gradient-based repair methods have been commonly introduced into constraint-handling techniques to handle linear and non-linear constraints. These gradient-based repair methods are only designed to reduce constraint violations, and they do not act on the objective function in the repairing process. Nevertheless, the gradient descent optimization method is originally proposed to optimize the objective function without constraints. Motivated by this consideration, this study develops an improved gradient-based repair method that incorporates the objective function to handle the constraints and optimize the objective function simultaneously. The proposed repair method is integrated into a multiobjective differential evolution framework to investigate its effectiveness. Experiments have been conducted on 57 real-world constrained benchmark test functions. The empirical result shows that, compared to the selected state-of-the-art algorithms, our proposed gradient-based repair method can assist the adopted constrained optimization approach to obtain high-quality feasible solutions.

Many real-world optimization problems are inherently constrained, presenting significant challenges to the application of evolutionary algorithms. Successfully managing these constraints while simultaneously optimizing the objective function requires a considerable degree of population diversity. To address this, we have developed a methodology that effectively combines an$\varepsilon$-constraint-handling method with a niching technique. The$\varepsilon$-constraint- handling method is specifically designed to manage constraints, while the niching technique aims to preserve pop-ulation diversity. In our approach, a constrained optimization problem is transformed into a tri-objective optimization challenge, introducing two additional objectives: the density objective and the overall constraint objective. The density objective is a particularly innovative aspect of our method, as it prolongs the survival of promising yet infeasible solutions. This prolongation aids the evolutionary search in converging towards the feasible region from various directions, thereby increasing the chances of identifying optimal solutions. Moreover, an improved gradient repair mutation strategy, based on a successful information reuse approach, is implemented to further refine promising solutions. To evaluate the effectiveness of our method, we tested it on 30 real-world constrained optimization problems from the CEC 2020 benchmark test suite. The results demonstrate that our approach either exceeds or is equivalent to the performance of current state-of-the-art constrained optimization algorithms.

In recent years, surrogate-assisted evolutionary algorithms have gained considerable success in addressing expensive constrained optimization problems. While significant focus has been directed toward optimization challenges with inequality constraints, the domain of expensive equality-constrained optimization also necessitates attention, as equality constraints are frequently encountered in traditional constrained optimization problems. Recognizing this gap, this study introduces an innovative approach that integrates a multilayer perceptron regression-based surrogate with a gradient descent-based repair method and differential evolution to address these challenges effectively. Our contributions are threefold: 1) We develop a multilayer perceptron-based surrogate model that concurrently approximates the objective function and equality constraints, 2) We employ a gradient descent-based repair method to adeptly manage the challenging equality constraints, and 3) We propose a hybrid local search scheme that enhances the solution refinement process. The combined use of the multilayer perceptron-based surrogate and gradient descent-based local search works in concert with differential evolution to guide the population toward the feasible region. This approach enables the evolutionary search, supported by the surrogate model, to extensively explore potential feasible regions. Our experimental results underscore the potential and efficacy of the proposed surrogate-assisted evolutionary algorithm in solving such complex optimization problems.

The recent proliferation of Internet of Things (IoT) sensors has driven a myriad of industrial and urban applications. Through analyzing massive data collected by these sensors, the proactive maintenance management can be achieved such that the maintenance schedule of the installed equipment can be optimized. Despite recent progress in proactive maintenance management in industrial scenarios, there are few studies on proactive maintenance management in urban informatics. In this article, we present an integrated framework of IoT and cloud computing platform for the proactive maintenance management in smart city. Our framework consists of: 1) an IoT monitoring system for collecting time-series data of operating and ambient conditions of the equipment and 2) a hybrid deep learning model, namely, convolutional bidirectional long short-term memory (CBLM) model for forecasting the operating and ambient conditions based on the collected time-series data. In addition, we also develop a naïve Bayes classifier to detect abnormal operating and ambient conditions and assist management personnel in scheduling maintenance tasks. To evaluate our framework, we deployed the IoT system in a Hong Kong public toilet, which is the first application of proactive maintenance management for a public hygiene and sanitary facility to the best of our knowledge. We collected the sensed data more than 33 days (808 h) in this real system. Extensive experiments on the collected data demonstrated that our proposed CBLM outperformed six traditional machine learning algorithms.

Genetic Programming is a method to automatically create computer programs based on the principles of evolution. The problem of deceptiveness caused by complex dependencies among components of programs is challenging. It is important because it can misguide Genetic Programming to create suboptimal programs. Besides, a minor modification in the programs may lead to a notable change in the program behaviours and affect the final outputs. This article presents Grammar-Based Genetic Programming with Bayesian Classifiers (GBGPBC) in which the probabilistic dependencies among components of programs are captured using a set of Bayesian network classifiers. Our system was evaluated using a set of benchmark problems (the deceptive maximum problems, the royal tree problems, and the bipolar asymmetric royal tree problems). It was shown to be often more robust and more efficient in searching the best programs than other related Genetic Programming approaches in terms of the total number of fitness evaluation. We studied what factors affect the performance of GBGPBC and discovered that robust variants of GBGPBC were consistently weakly correlated with some complexity measures. Furthermore, our approach has been applied to learn a ranking program on a set of customers in direct marketing. Our suggested solutions help companies to earn significantly more when compared with other solutions produced by several well-known machine learning algorithms, such as neural networks, logistic regression, and Bayesian networks.

Financial fraud is a criminal act, which violates the law, rules or policy to gain unauthorized financial benefit. The major consequences are loss of billions of dollars each year, investor confidence or corporate reputation. A study area called Financial Fraud Detection (FFD) is obligatory, in order to prevent the destructive results caused by financial fraud. In this study, we propose a new method based on Grammar-based Genetic Programming (GBGP), multi-objectives optimization and ensemble learning for solving FFD problems. We comprehensively compare the proposed method with Logistic Regression (LR), Neural Networks (NNs), Support Vector Machine (SVM), Bayesian Networks (BNs), Decision Trees (DTs), AdaBoost, Bagging and LogitBoost on four FFD datasets. The experimental results showed the effectiveness of the new approach in the given FFD problems including two real-life problems. The major implications and significances of the study can concretely generalize for two points. First, it evaluates a number of data mining techniques by the given real-life classification problems. Second, it suggests a new method based on GBGP, NSGA-II and ensemble learning.

Gene regulatory network (GRN), which refers to the complex interactions with time delays between TFs and other genes, plays an important role in the working of the cell. Therefore inferring the GRN is crucial to studying diseases related to malfunctioning of the cell. Even with high-throughput technology, time series expression data is still limited compared to the network size, which poses significant challenge to inferring large GRN. Since GRNs are known to be modular, or hierarchically modular, we propose to exploit this by first inferring an initial GRN using CLINDE, then decomposing it into possibly overlapping subnetworks, then re-learning the subnetworks using either CLINDE or DD-lasso, and lastly merging the subnetworks. We have performed extensive experiments on synthetic data to test this strategy on both modular and hierarchically modular networks with 500 and 1000 genes, using either a long time series or several short time series. Results show that the strategy does improve GRN inference with statistical significance. Also, the algorithm is robust to different variance and slight deviation of Gaussianity for the error terms.

BACKGROUND: Inferring gene regulatory network (GRN) has been an important topic in Bioinformatics. Many computational methods infer the GRN from high-throughput expression data. Due to the presence of time delays in the regulatory relationships, High-Order Dynamic Bayesian Network (HO-DBN) is a good model of GRN. However, previous GRN inference methods assume causal sufficiency, i.e. no unobserved common cause. This assumption is convenient but unrealistic, because it is possible that relevant factors have not even been conceived of and therefore un-measured. Therefore an inference method that also handles hidden common cause(s) is highly desirable. Also, previous methods for discovering hidden common causes either do not handle multi-step time delays or restrict that the parents of hidden common causes are not observed genes. RESULTS: We have developed a discrete HO-DBN learning algorithm that can infer also hidden common cause(s) from discrete time series expression data, with some assumptions on the conditional distribution, but is less restrictive than previous methods. We assume that each hidden variable has only observed variables as children and parents, with at least two children and possibly no parents. We also make the simplifying assumption that children of hidden variable(s) are not linked to each other. Moreover, our proposed algorithm can also utilize multiple short time series (not necessarily of the same length), as long time series are difficult to obtain. CONCLUSIONS: We have performed extensive experiments using synthetic data on GRNs of size up to 100, with up to 10 hidden nodes. Experiment results show that our proposed algorithm can recover the causal GRNs adequately given the incomplete data. Using the limited real expression data and small subnetworks of the YEASTRACT network, we have also demonstrated the potential of our algorithm on real data, though more time series expression data is needed.

Grammar-Based Genetic Programming (GBGP) improves the search performance of Genetic Programming (GP) by formalizing constraints and domain specific knowledge in grammar. The building blocks (i.e. the functions and the terminals) in a program can be dependent. Random crossover and mutation destroy the dependence with a high probability, hence breeding a poor program from good programs. Understanding on the syntactic and semantic in the grammar plays an important role to boost the efficiency of GP by reducing the number of poor breeding. Therefore, approaches have been proposed by introducing context sensitive ingredients encoded in probabilistic models. In this paper, we propose Grammar-Based Genetic Programming with Bayesian Network (BGBGP) which learns the dependence by attaching a Bayesian network to each derivation rule and demonstrates its effectiveness in two benchmark problems.

Grammar-Based Genetic Programming formalizes constraints on the solution structure based on domain knowledge to reduce the search space and generate grammatically correct individuals. Nevertheless, building blocks in a program can often be dependent, so the effective search space can be further reduced. Approaches have been proposed to learn the dependence using probabilistic models and shown to be useful in finding the optimal solutions with complex structure. It raises questions on how to use the individuals in the population to uncover the underlying dependence. Usually, only the good individuals are selected. To model the dependence better, we introduce Grammar-Based Genetic Programming with Bayesian Network Classifier (GBGPBC) which also uses poorer individuals. With the introduction of class labels, we further propose a refinement technique on probability distribution based on class label. Our results show that GBGPBC performs well on two benchmark problems. These techniques boost the performance of our system.

DNA motif discovery is an important problem for deciphering protein-DNA bindings in gene regulation. To discover generic spaced motifs which have multiple conserved patterns separated by wild-cards called spacers, the genetic algorithm (GA) based GASMEN has been proposed and shown to outperform related methods. However, the over-generic modeling of any number of spacers increases the optimization difficulty in practice. In protein-DNA binding case studies, complicated spaced motifs are rare while dimers with single spacers are more common spaced motifs. Moreover, errors (mismatches) in a conserved pattern are not arbitrarily distributed as certain highly conserved nucleotides are essential to maintain bindings. Motivated by better optimization in real applications, we have developed a new method, which is GA for Dimer-led and Error-restricted Spaced Motifs (GADESM). Common spaced motifs are paid special attention to using dimer-led initialization in the population initialization. The results on real datasets show that the dimer-led initialization in GADESM achieves better fitness than GASMEN with statistical significance. With additional error-restricted motif occurrence retrieval, GADESM has shown better performance than GASMEN on both comprehensive simulation data and a real ChIP-seq case study.

An important and challenging data mining application in marketing is to learn models for predicting potential customers who contribute large profit to a company under resource constraints. In this paper, we first formulate this learning problem as a constrained optimization problem and then converse it to an unconstrained Multi-objective Optimization Problem (MOP). A parallel Multi-Objective Evolutionary Algorithm (MOEA) on consumer-level graphics hardware is used to handle the MOP. We perform experiments on a real-life direct marketing problem to compare the proposed method with the parallel Hybrid Genetic Algorithm, the DMAX approach, and a sequential MOEA. It is observed that the proposed method is much more effective and efficient than the other approaches.

Multi-class problem is the class of problems having more than one classes in the data set. Bayesian Network (BN) is a well-known algorithm handling the multi-class problem and is applied to different areas. But BN cannot handle continuous values. In contrast, Genetic Programming (GP) can handle continuous values and produces classification rules. However, GP is possible to produce cyclic rules representing tautologic, in which are useless for inference and expert systems. Co-evolutionary Rule-chaining Genetic Programming (CRGP) is the first variant of GP handling the multi-class problem and produces acyclic classification rules [16]. It employs backward chaining inference to carry out classification based on the acquired acyclic rule set. It can handle multi-classes; it can avoid cyclic rules; it can handle input attributes with continuous values; and it can learn complex relationships among the attributes. In this paper, we propose a novel algorithm, the Chaining Genetic Programming (CGP) learning a set of acyclic rules and to produce better results than the CRGP's. The experimental results demonstrate that the proposed algorithm has the shorter learning process and can produce more accurate acyclic classification rules.

Existing Structural Expectation-Maximization (EM) algorithms for learning Bayesian networks from incomplete data usually adopt the greedy hill climbing search method, which may make the algorithms find sub-optimal solutions. In this paper, we present a new Structural EM algorithm which employs a hybrid evolutionary algorithm as the search method. The experimental results on the data sets generated from several benchmark networks illustrate that our algorithm outperforms some state-of-the-art learning algorithms.

Classification rule is a useful model in data mining. Given variable values, rules classify data items into different classes. Different rule learning algorithms are proposed, like Genetic Algorithm (GA) and Genetic Programming (GP). Rules can also be extracted from Bayesian Network (BN) and decision trees. However, all of them have disadvantages and may fail to get the best results. Both of GA and GP cannot handle cooperation among rules and thus, the learnt rules are likely to have many overlappings, i. e. more than one rules classify the same data items and different rules have different predictions. The conflicts among the rules reduce their understandability and increase their usage difficulty for expert systems. In contrast, rules extracted from BN and decision trees have no overlapping in nature. But BN can handle discrete values only and cannot represent higher-order relationships among variables. Moreover, the search space for decision tree learning is huge and thus, it is difficult to reach the global optimum. In this paper, we propose to use Functional Dependency Network (FDN) and MDL Genetic Programming (MDLGP) to learn a set of non-overlapping classification rules [17]. The FDN is an extension of BN; it can handle all kind of values; it can represent higher-order relationships among variables; and its learning search space is smaller than decision trees’. The experimental results demonstrate that the proposed method can successfully discover the target rules, which have no overlapping and have the highest classification accuracies.

In this paper, we report a parallel Hybrid Genetic Algorithm (HGA) on consumer-level graphics cards. HGA extends the classical genetic algorithm by incorporating the Cauchy mutation operator from evolutionary programming. In our parallel HGA, all steps except the random number generation procedure are performed in Graphics Processing Unit (GPU) and thus our parallel HGA can be executed effectively and efficiently. We propose the pseudo-deterministic selection method which is comparable to the traditional global selection approach with significant execution time performance advantages. We perform experiments to compare our parallel HGA with our previous parallel FEP (Fast Evolutionary programming) and demonstrate that the former is much more effective and efficient than the latter. The parallel and sequential implementations of HGA are compared in a number of experiments, it is observed that the former outperforms the latter significantly. The effectiveness and efficiency of the pseudo-deterministic selection method is also studied.

This paper proposes a novel hybrid approach for learning Bayesian networks from incomplete data in the presence of missing values, which combines an evolutionary algorithm with the traditional expectation-maximization (EM) algorithm. The new algorithm can overcome the problem of getting stuck in sub-optimal solutions which occurs in most existing learning algorithms. The experimental results on the data sets generated from several benchmark networks illustrate that the new algorithm has better performance than some state-of-the-art algorithms. We also apply the approach to a data set of direct marketing and compare the performance of the discovered Bayesian networks obtained by the new algorithm with the networks generated by other methods. In the comparison, the Bayesian networks learned by the new algorithm outperform other networks.

Evolutionary algorithms (EAs) are effective and robust methods for solving many practical problems such as feature selection, electrical circuit synthesis, and data mining. However, they may execute for a long time for some difficult problems, because several fitness evaluations must be performed. A promising approach to overcome this limitation is to parallelize these algorithms. In this paper, we propose to implement a parallel EA on consumer-level graphics cards. We perform experiments to compare our parallel EA with an ordinary EA and demonstrate that the former is much more effective than the latter. Since consumer-level graphics cards are available in ubiquitous personal computers and these computers are easy to use and manage, more people are able to use our parallel algorithm to solve their problems encountered in real-world applications.

Bayesian Network (BN) is a powerful network model, which represents a set of variables in the domain and provides the probabilistic relationships among them. But BN can handle discrete values only; it cannot handle continuous, interval and ordinal ones, which must be converted to discrete values and the order information is lost. Thus, BN tends to have higher network complexity and lower understandability. In this paper, we present a novel dependency network which can handle discrete, continuous, interval and ordinal values through functions; it has lower network complexity and stronger expressive power; it can represent any kind of relationships; and it can incorporate a-priori knowledge though user-defined functions. We also propose a novel Genetic Programming (GP) to learn dependency networks. The novel GP does not use any knowledge-guided nor application-oriented operator, thus it is robust and easy to replicate. The experimental results demonstrate that the novel GP can successfully discover the target novel dependency networks, which have the highest accuracy and the lowest network complexity.

The scalability problem in data mining involves the development of methods for handling large databases with limited computational resources such as memory and computation time. In this paper, two scalable clustering algorithms, bEMADS and gEMADS, are presented based on the Gaussian mixture model. Both summarize data into subclusters and then generate Gaussian mixtures from their data summaries. Their core algorithm, EMADS, is defined on data summaries and approximates the aggregate behavior of each subcluster of data under the Gaussian mixture model. EMADS is provably convergent. Experimental results substantiate that both algorithms can run several orders of magnitude faster than expectation-maximization with little loss of accuracy.

Given the explosive growth of data collected from current business environment, data mining can potentially discover new knowledge to improve managerial decision making. This paper proposes a novel data mining approach that employs an evolutionary algorithm to discover knowledge represented in Bayesian networks. The approach is applied successfully to handle the business problem of finding response models from direct marketing data. Learning Bayesian networks from data is a difficult problem. There are two different approaches to the network learning problem. The first one uses dependency analysis, while the second one searches good network structures according to a metric. Unfortunately, both approaches have their own drawbacks. Thus, we propose a novel hybrid algorithm of the two approaches, which consists of two phases, namely, the conditional independence (CI) test and the search phases. In the CI test phase, dependency analysis is conducted to reduce the size of the search space. In the search phase, good Bayesian network models are generated by using an evolutionary algorithm. A new operator is introduced to further enhance the search effectiveness and efficiency. In a number of experiments and comparisons, the hybrid algorithm outperforms MDLEP, our previous algorithm which uses evolutionary programming (EP) for network learning, and other network learning algorithms. We then apply the approach to two data sets of direct marketing and compare the performance of the evolved Bayesian networks obtained by the new algorithm with those by MDLEP, the logistic regression models, the na/spl inodot//spl uml/ve Bayesian classifiers, and the tree-augmented na/spl inodot//spl uml/ve Bayesian network classifiers (TAN). In the comparison, the new algorithm outperforms the others.

The issue of obtaining a well-converged and well-distributed set of Pareto optimal solutions efficiently and automatically is crucial in multiobjective evolutionary algorithms (MOEAs). Many studies have proposed different evolutionary algorithms that can progress towards Pareto optimal sets with a wide-spread distribution of solutions. However, most mathematically convergent MOEAs desire certain prior knowledge about the objective space in order to efficiently maintain widespread solutions. We propose, based on our novel E-dominance concept, an adaptive rectangle archiving (ARA) strategy that overcomes this important problem. The MOEA with this archiving technique provably converges to well-distributed Pareto optimal solutions without prior knowledge. ARA complements the existing archiving techniques, and is useful to both researchers and practitioners.

The scalability problem in data mining involves the development of methods for handling large databases with limited computational resources. We present a two-phase scalable model-based clustering framework: first, a large data set is summed up into subclusters; Then, clusters are directly generated from the summary statistics of subclusters by a specifically designed expectation-maximization (EM) algorithm. Taking example for Gaussian mixture models, we establish a provably convergent EM algorithm, EMADS, which embodies cardinality, mean, and covariance information of each subcluster explicitly. Combining with different data summarization procedures, EMADS is used to construct two clustering systems: gEMADS and bEMADS. The experimental results demonstrate that they run several orders of magnitude faster than the classic EM algorithm with little loss of accuracy. They generate significantly better results than other model-based clustering systems using similar computational resources.

As a typical combinatorial optimization problem, the traveling salesman problem (TSP) has attracted extensive research interest. In this paper, we develop a self-organizing map (SOM) with a novel learning rule. It is called the integrated SOM (ISOM) since its learning rule integrates the three learning mechanisms in the SOM literature. Within a single learning step, the excited neuron is first dragged toward the input city, then pushed to the convex hull of the TSP, and finally drawn toward the middle point of its two neighboring neurons. A genetic algorithm is successfully specified to determine the elaborate coordination among the three learning mechanisms as well as the suitable parameter setting. The evolved ISOM (eISOM) is examined on three sets of TSP to demonstrate its power and efficiency. The computation complexity of the eISOM is quadratic, which is comparable to other SOM-like neural networks. Moreover, the eISOM can generate more accurate solutions than several typical approaches for TSP including the SOM developed by Budinich, the expanding SOM, the convex elastic net, and the FLEXMAP algorithm. Though its solution accuracy is not yet comparable to some sophisticated heuristics, the eISOM is one of the most accurate neural networks for the TSP.

A novel hybrid framework is reported that improves upon our previous work, MDLEP, which uses evolutionary programming to solve the difficult Bayesian network learning problem. A new merge operator is also introduced that further enhances the efficiency. As experimental results suggest, our hybrid approach performs significantly better than MDLEP.

The self-organizing map (SOM) is a powerful tool in the exploratory phase of data mining. However, due to the dimensional conflict, neighborhood preservation cannot always lead to perfect topology preservation. In this paper we establish an expanding SOM (ESOM) to detect and preserve better topology correspondence between the two spaces. Our experiment results demonstrate that the ESOM constructs better mappings than the classic SOM in terms of both topological and quantization errors. Furthermore, clustering results generated by the ESOM are more accurate than those of the SOM.

Describes a data mining approach that employs evolutionary programming to discover knowledge represented in Bayesian networks. There are two different approaches to the network learning problem. The first one uses dependency analysis, while the second one searches good network structures according to a metric. Unfortunately, both approaches have their own drawbacks. Thus, we propose a hybrid algorithm of the two approaches, which consists of two phases, namely, the conditional independence test and the search phases. A new operator is introduced to further enhance the search efficiency. We conduct a number of experiments and compare the hybrid algorithm with our previous algorithm, MDLEP, which uses EP for network learning. The empirical results illustrate that the new approach has better performance. We apply the approach to data sets of direct marketing and compare the performance of the evolved Bayesian networks obtained by the new algorithm with the models generated by other methods. In the comparison, the induced Bayesian networks produced by the new algorithm outperform the other models.

This paper describes a novel knowledge discovery and data mining framework dealing with nonlinear interactions among domain attributes. Our network-based model provides an effective and efficient reasoning procedure to perform prediction and decision making. Unlike many existing paradigms based on linear models, the attribute relationship in our framework is represented by nonlinear nonnegative multiregressions based on the Choquet integral. This kind of multiregression is able to model a rich set of nonlinear interactions directly. Our framework involves two layers. The outer layer is a network structure consisting of network elements as its components, while the inner layer is concerned with a particular network element modeled by Choquet integrals. We develop a fast double optimization algorithm (FDOA) for learning the multiregression coefficients of a single network element. Using this local learning component and multiregression-residual-cost evolutionary programming (MRCEP), we propose a global learning algorithm, called MRCEP-FDOA, for discovering the network structures and their elements from databases. We have conducted a series of experiments to assess the effectiveness of our algorithm and investigate the performance under different parameter combinations, as well as sizes of the training data sets. The empirical results demonstrate that our framework can successfully discover the target network structure and the regression coefficients.

Multiregression is one of the most common approaches used to discover dependency pattern among attributes in a database. Nonadditive set functions have been applied to deal with the interactive predictive attributes involved, and some nonlinear integrals with respect to nonadditive set functions are employed to establish a nonlinear multiregression model describing the relation between the objective attribute and predictive attributes. The values of the nonadditive set function play a role of unknown regression coefficients in the model and are determined by an adaptive genetic algorithm from the data of predictive and objective attributes. Furthermore, such a model is now improved by a new numericalization technique such that the model can accommodate both categorical and continuous numerical attributes. The traditional dummy binary method dealing with the mixed type data can be regarded as a very special case of our model when there is no interaction among the predictive attributes and the Choquet integral is used. When running the algorithm, to avoid a premature during the evolutionary procedure, a technique of maintaining diversity in the population is adopted. A test example shows that the algorithm and the relevant program have a good reversibility for the data. © 2001 John Wiley & Sons, Inc.16: 949–962 (2001)

In data mining, we emphasize the need for learning from huge, incomplete, and imperfect data sets. To handle noise in the problem domain, existing learning systems avoid overfitting the imperfect training examples by excluding insignificant patterns. The problem is that these systems use a limiting attribute-value language for representing the training examples and the induced knowledge. Moreover, some important patterns are ignored because they are statistically insignificant. In this article, we present a framework that combines Genetic Programming and Inductive Logic Programming to induce knowledge represented in various knowledge representation formalisms from noisy databases. The framework is based on a formalism of logic grammars, and it can specify the search space declaratively. An implementation of the framework, LOGENPRO (The Logic grammar based GENetic PROgramming system), has been developed. The performance of LOGENPRO is evaluated on the chess end-game domain. We compare LOGENPRO with FOIL and other learning systems in detail, and find its performance is significantly better than that of the others. This result indicates that the Darwinian principle of natural selection is a plausible noise handling method that can avoid overfitting and identify important patterns at the same time. Moreover, the system is applied to one real-life medical database. The knowledge discovered provides insights to and allows better understanding of the medical domains.

We have developed a new approach to learning Bayesian network structures based on the minimum description length (MDL) principle and evolutionary programming. It employs a MDL metric, which is founded on information theory, and integrates a knowledge-guided genetic operator for the optimization in the search process.

By using a non-additive set function to describe the interaction among variables, a nonlinear non-negative multi-regression is established based on the Choquet integral with respect to the set function. We generalize this nonlinear model and propose a novel formalism that provides an effective and efficient reasoning procedure to perform information fusion, decision making, and medical diagnoses. In the formalism, a network structure and a number of Choquet integrals are used to represent the relationships among variables. We propose a new algorithm to learn the network structure and the regression parameters of Choquet integrals from training examples in databases. The algorithm is based on the minimum description length (MDL) principle and evolutionary programming (EP). We conduct a series of experiments to demonstrate the performance of our algorithm and estimate the effectiveness of the MDL metric and the genetic operators. The empirical results illustrate that our algorithm can successfully discover the target network structure and the regression parameter.

Program induction generates a computer program that can produce the desired behavior for a given set of situations. Two of the approaches in program induction are inductive logic programming (ILP) and genetic programming (GP). Since their formalisms are so different, these two approaches cannot be integrated easily, although they share many common goals and functionalities. A unification will greatly enhance their problem-solving power. Moreover, they are restricted in the computer languages in which programs can be induced. In this paper, we present a flexible system called LOGENPRO (The LOgic gramar-based GENetic PROgramming system) that uses some of the techniques of GP and ILP. It is based on a formalism of logic grammars. The system applies logic grammars to control the evolution of programs in various programming languages and represent context-sensitive information and domain-dependent knowledge. Experiments have been performed to demonstrate that LOGENPRO can emulate GP and GP with automatically defined functions (ADFs). Moreover, LOGENPRO can employ knowledge such as argument types in a unified framework. The experiments show that LOGENPRO has superior performance to that of GP and GP with ADFs when more domain-dependent knowledge is available. We have applied LOGENPRO to evolve general recursive functions for the even-n-parity problem from noisy training examples. A number of experiments have been performed to determine the impact of domain-specific knowledge and noise in training examples on the speed of learning.

Genetic programming (GP) and inductive logic programming (ILP) have received increasing interest. Since their formalisms are so different these two approaches cannot be integrated easily though they share many common goals and functionalities. A unification will greatly enhance their problem solving power. Moreover, they are restricted in the computer languages in which programs can be induced. We present a flexible system called LOGENPRO (The logic grammar based genetic programming system) that combines GP and ILP. It is based on a formalism of logic grammars. The system can learn programs in various programming languages and represent context-sensitive information and domain-dependent knowledge. The performance of LOGENPRO in inducing logic programs from noisy examples is evaluated. A detailed comparison with FOIL has been conducted. This experiment demonstrates that LOGENPRO is a promising alternative to other inductive logic programming systems and sometimes is superior for handling noisy data. Moreover, a series of examples are used to illustrate that LOGENPRO is so flexible that programs in different programming languages including LISP, Prolog and Fuzzy Prolog can be induced.