With the rapid adoption of large language models (LLMs) in natural language processing, the ability to follow instructions has emerged as a key metric for evaluating their practical utility.However, existing evaluation methods often focus on single-language scenarios, overlooking the challenges and differences present in multilingual and cross-lingual contexts.To address this gap, we introduce MaXIFE: a comprehensive evaluation benchmark designed to assess instruction-following capabilities across 23 different languages with 1667 verifiable instruction tasks.MaXIFE integrates both Rule-Based Evaluation and Model-Based Evaluation, ensuring a balance of efficiency and accuracy.We applied MaXIFE to evaluate several leading commercial LLMs, establishing baseline results for future comparisons.By providing a standardized tool for multilingual instructionfollowing evaluation, MaXIFE aims to advance research and development in natural language processing.

The pedestrian trajectory prediction is a crucial research topic in artificial intelligence application scenarios like autopilot and robotics. In these kinds of scenarios, the autopilot vehicle or robot should have a cautious interaction with human to avoid accident. Over the past decade, researchers have continuously proposed high-performance pedestrian trajectory prediction methods by leveraging the powerful tool of artificial intelligence. In particular, the spatial-temporal features based methods have been successfully applied. However, one potential issue with spatial-temporal features has been overlooked. Due to the sensitivity inherent in pedestrian dataset collection, the diversity of spatial features is far less than that of temporal features. Therefore, most spatial-temporal features based methods are tend to overfit to scenarios features, results in an unstable results across different scenarios. In our work, a Kinematic Temporal Conditional Variational Autoencoder (KT-VAE) that emphasizes the importance of temporal features along with a reliable spatial post-processing method is proposed. In KT-VAE, the spatial features are compressed instead of the temporal features to ensure that the model focuses more on the temporal continuity of pedestrian kinematic. This approach enables the VAE to better capture the temporal continuity and dynamic characteristics of pedestrian motion, while avoiding scenario overfitting that can result from insufficient spatial features. Through experiments, the KT-VAE maintains stability across different scenarios in cross-validation and demonstrates competitive performance in practical applications.

Music generation task is commonly considered as a note-by-note prediction problem. Moreover, prediction models generating one musical note at a time may ignore the overall coherence because the music phrase is incomplete and unable to demonstrate musicality. To address these issues, in this study, we propose a feasible monophonic music generation framework that can simulate subsequent trends for each predicted musical note. The framework generates a musical note mainly in three steps: 1) a sequence prediction model is used to predict the most potential candidates, 2) the subsequent trends for each candidate are modeled and evaluated, and 3) the best candidate is selected as the final result. We use the Monte-Carlo tree search algorithm because of its great capability of discovering near-optimal results. We establish a method of training a value network that can assess musical coherence to evaluate the simulated sequences. Further, we used a smoothed polynomial upper confidence trees algorithm to improve the accuracy and efficiency of the search process. An accurate dataset labeled by us, which contains 36 transcribed samples from real-world pop songs, was used to validate our framework. Compared with the note-by-note sequence prediction model, our framework exhibits a better sense of musicality. Our framework can be applied to generate symbolic monophonic music, particularly the main melody track in pop music.

Few-shot image classification has attracted much attention due to its requirement of limited training data for target classes. Existing methods usually pretrain a network with images from the base set as feature extractor to obtain features of images from novel set. However, the pretrained feature extractor cannot extract accurate representation for categories have never seen, making images from novel set difficult to distinguish. To be specific, in the pretrained feature space, there exist a large number of overlapped areas between novel categories. To address this issue, it is crucial to acquire a space, where features from same class are gathering together and features from different classes are far away from each other. Since lots of experiments have proved that the triplet network is effective to achieve this goal, in this paper, we base our network on the Maximum a posteriori (MAP), learning a latent space with triplet network to project features from pretrained feature space into a more discriminative one. Experimental results on four few-shot benchmarks show that it significantly outperforms the baseline methods, improves around 1.09%∼13.09% than the best results in each dataset on both 1- and 5-shot tasks.

The temporal order of user behaviors, which implies the user's preference in the near future, plays a key role in sequential recommendation systems. To capture such patterns from user behavior sequences, many recent works borrow ideas from language models and consider it a next item prediction problem. It is reasonable, but the gap between the user behavior data and the text data is ignored. Generally speaking, user behaviors are more arbitrary than sentences in natural languages. A behavior sequence usually carries multiple intentions, and the exact order does not matter a lot. But a sentence in a text tends to express one meaning and different orders of the words may bring very different meanings. To address these issues, this study considers user behavior as a mixture of multiple subsequences. Specifically, we introduce a subsequence extraction module, which assigns the items in a sequence into different subsequences, with respect to their relationship. Then these subsequences are fed into the downstream sequence model, from which we obtain several user representations. To train the whole system in an end-to-end manner, we design a new training strategy where only the user representation near the target item gets supervised. To verify the effectiveness of our method, we conduct extensive experiments on four public datasets. It is compared with several baselines and achieves better results in most cases. Further experiments explore the properties of our model and we also visualize the result of the subsequence extraction.

Commonly music has an obvious hierarchical structure, especially for the singing parts which usually act as the main melody in pop songs. However, most of the current singing annotation datasets only record symbolic information of music notes, ignoring the structure of music. In this paper, we propose a hierarchical singing annotation dataset that consists of 68 pop songs from Youtube. This dataset records the onset/offset time, pitch, duration, and lyric of each musical note in an enhanced LyRiCs (LRC) format to present the hierarchical structure of music. We annotate each song in a two-stage process: first, create initial labels with the corresponding musical notation and lyrics file; second, manually calibrate these labels referring to the raw audio. We mainly validate the labeling accuracy of the proposed dataset by comparing it with an automatic singing transcription (AST) dataset. The result indicates that the proposed dataset reaches the labeling accuracy of AST datasets.

Abstract Few‐shot learning is to distinguish novel concepts with few annotated data, which has attracted much attention due to its requirement of limited training data for target classes. Recent few‐shot learning methods usually pretrain a feature extractor with images from the base set to boost the performance of few‐shot tasks and classify novel categories in this prior feature space. However, it is difficult for the pretrained feature extractor to extract accurate representations for novel categories, resulting in large amounts of overlapping areas between new classes. To address these issues, the prior feature space with a triplet network to learn a more discriminative space is refined, where features belonging to same class are pulled together and that from different classes are pushed apart. Specifically, the authors first follow recent paradigm of pretraining to obtain a prior feature space. Then, a triplet network with contrastive learning is trained to project the features from this space into a low‐dimensional one. The main difference lies in that the authors’ model is based on Maximum A Posteriori (MAP) and the triplet network with hallucinated features is finetuned from it to make them generalise well to novel categories. Finally, the authors conduct classification tasks in the finetuned space. The authors’ intuition is that the overlapping areas in novel categories can be separated by finetuning the triplet network pretrained on base set with contrastive learning. Experimental results on four few‐shot benchmarks show that it significantly outperforms the baseline methods, improves around 1.09% ∼ 13.09% than the best results in each dataset on both 1‐ and 5‐shot tasks.

To develop an efficient brain-computer interface (BCI) system, electroencephalography (EEG) measures neuronal activities in different brain regions through electrodes. Many EEG-based motor imagery (MI) studies do not make full use of brain network topology. In this paper, a deep learning framework based on a modified graph convolution neural network (M-GCN) is proposed, in which temporal-frequency processing is performed on the data through modified S-transform (MST) to improve the decoding performance of original EEG signals in different types of MI recognition. MST can be matched with the spatial position relationship of the electrodes. This method fusions multiple features in the temporal-frequency-spatial domain to further improve the recognition performance. By detecting the brain function characteristics of each specific rhythm, EEG generated by imaginary movement can be effectively analyzed to obtain the subjects' intention. Finally, the EEG signals of patients with spinal cord injury (SCI) are used to establish a correlation matrix containing EEG channel information, the M-GCN is employed to decode relation features. The proposed M-GCN framework has better performance than other existing methods. The accuracy of classifying and identifying MI tasks through the M-GCN method can reach 87.456%. After 10-fold cross-validation, the average accuracy rate is 87.442%, which verifies the reliability and stability of the proposed algorithm. Furthermore, the method provides effective rehabilitation training for patients with SCI to partially restore motor function.

Recently, deep learning based video object detection has attracted more and more attention. Compared with object detection of static images, video object detection is more challenging due to the motion of objects, while providing rich temporal information. The RNN-based algorithm is an effective way to enhance detection performance in videos with temporal information. However, most studies in this area only focus on accuracy while ignoring the calculation cost and the number of parameters.In this paper, we propose an efficient method that combines channel-reduced convolutional GRU (Squeezed – GRU), and Information Entropy map for video object detection (SGE-Net). The experimental results validate the accuracy improvement, computational savings of the Squeezed GRU, and superiority of the information entropy attention mechanism on the classification performance. The mAP has increased by 3.7 contrasted with the baseline, and the number of parameters has decreased from 6.33 million to 0.67 million compared with the standard GRU.

Deep reinforcement learning algorithms have made great progress in video games. However, there are still some problems, such as sample inefficiency and poor generalization. In this paper, we highlight that these problems are partially caused by the inability of convolutional neural networks (CNNs) to reason with the underlying relations between the objects in the image observations. Based on this point, we try to alleviate these problems in a more efficient and explainable way, including learning the representations of objects and reasoning the relations between them with a relation network (RN). Each pixel in the feature maps is treated as an object and our model explicitly learns the relations between object pairs. The relations are summarized through an attention mechanism and then fed into the downstream fully-connected layers. In the experiments, our model is compared with baseline models in three typical object based Atari games. Under the same hyperparameter settings, our model still achieves better sample efficiency and generalization capability. Further studies throw light on the impact of hyperparameters and verify the interpretability of the model.

A common shortfall of supervised learning for medical imaging is the greedy need for human annotations, which is often expensive and time-consuming to obtain. This paper proposes a semi-supervised classification method for three kinds of apicomplexan parasites and non-infected host cells microscopic images, which uses a small number of labeled data and a large number of unlabeled data for training. There are two challenges in microscopic image recognition. The first is that salient structures of the microscopic images are more fuzzy and intricate than natural images’ on a real-world scale. The second is that insignificant textures, like background staining, lightness, and contrast level, vary a lot in samples from different clinical scenarios. To address these challenges, we aim to learn a distinguishable and appearance-invariant representation by contrastive learning strategy. On one hand, macroscopic images, which share similar shape characteristics in morphology, are introduced to contrast for structure enhancement. On the other hand, different appearance transformations, including color distortion and flittering, are utilized to contrast for texture elimination. In the case where only 1% of microscopic images are labeled, the proposed method reaches an accuracy of 94.90% in a generalized testing set.

This paper deals with the problem of detecting protein complexes from protein-protein interaction (PPI) network using a spectral clustering method. A complete PPI network is crucial for detection performance. However, experimentally identified PPIs are usually very limited, resulting in incomplete PPI networks. To solve this problem, we propose a deep transfer learning based predictor for the PPI prediction, consisting of a semi-supervised SVM classifier and a deep feature extractor of convolution neural network (CNN). Considering the fact that the similarities of gene ontology (GO) annotations contribute to protein interaction, and the difference of subcellular localizations contribute to negative interactions, we pre-train the deep CNN feature extractor in deep GO annotation and subcellular localization predictors and then transfer it to the PPI prediction. In this way, we have a deep PPI detector enhanced with transfer learning of GO annotation and subcellular localization prediction. Experimental results show that the proposed method outperforms the state-of-the-art methods on benchmark datasets.

Air pollution has threatened people’s health. It is urgent for the government to strengthen and improve the ability of air pollution monitoring. This paper proposes a winner-take-all (WTA) autoencoder-based piecewise linear model for imputing air quality prediction under the missing data scenario. The main idea consists of two parts. Firstly, overcomplete WTA stacked denoising autoencoders (SDAEs) are proposed to handle missing data, which play two roles: 1) devise the multiple imputation strategy to fill in missing values; 2) generate a set of binary gate control sequences to construct the sophisticated partitioning. Besides, renewed teacher signals are updated based on clustering information in the trained SDAEs to improve the accuracy of filling in missing samples. Secondly, the piecewise linear model is then proposed by the generated set of gate signals using the information from the feature layers of SDAEs. By using a quasi-linear kernel based on the trained gating mechanism, our piecewise air quality predictor is finally identified in the exact same way as support vector regression. The proposed modeling method is applied to real air quality datasets to show that it has led to greater performance than traditional models.

Evaluating image captions is very challenging partially due to the fact that there are multiple correct captions for every single image. Most of the existing one-to-one metrics operate by penalizing mismatches between reference and generative caption without considering the intrinsic variance between ground truth captions. It usually leads to over-penalization and thus a bad correlation to human judgment. Recently, the latest one-to-one metric BERTScore can achieve high human correlation in system-level tasks while some issues can be fixed for better performance. In this paper, we propose a novel metric based on BERTScore that could handle such a challenge and extend BERTScore with a few new features appropriately for image captioning evaluation. The experimental results show that our metric achieves state-of-the-art human judgment correlation.

In online advertising systems, having a good knowledge of user behavior is crucial for click-through rate (CTR) prediction. In recent years, many researchers turn to seek a better way of user representation by modeling the behavior sequences with recurrent neural network (RNN). However, recurrent layers implicitly adopt the assumption that elements with different orders are fundamentally different, which is inefficient in many practical scenarios with much uncertainty and complicated hidden states. In this paper, we follow the paradigm of Relation Network (RN), and propose a new model called Similitude Attentive Relation Network (SARN). The user behavior is modeled as a graph, where nodes correspond to the visited items and edges correspond to the relations. To capture the latent user interest better, the model concentrates on the relations between items, rather than the translation on the time series. More specifically, the model tries to learn the similarity between items in a semantic space through a learnable dot-product operation and blend both of the item representations and relational information together as the final relations. We define our user representation on an attentive pooling of the relations directly. To verify the effectiveness of our method, extensive experiments on two public datasets and one real-world online advertising dataset are conducted. Experimental results show that our methods achieve usually better performance than others. Besides, we explore the properties of our model by controlled experiments and show the learned relational knowledge by visualizing the inner states of SARN.

Semi-supervised learning aims to construct a classifier by making use of both labeled data and unlabeled data. This paper proposes a semi-supervised classification method using a gated linear model, based on the idea of effectively utilizing manifold information. A gating mechanism is firstly trained in a semi-supervised manner to capture manifold information which guides the generation of gate signals. Then the gated linear model is formulated into a linear regression form with the gate signals included. Secondly, a Laplacian regularized least squares (LapRLS) formulation is applied to optimize the linear regression form of the gated linear model. In this way, the gate signals are integrated into the kernel function, which is defined as inner product of the regression vectors. Moreover, this kernel function is used as a better similarity function for graph construction. As a result, the manifold information is ingeniously incorporated into both kernel and graph Laplacian in the LapRLS. Experimental results exhibit the effectiveness of our proposed method.

Predicting protein functions is a fundamental task with applications in medicine and healthcare. However, the accelerating pace of protein-discovery renders slow and expensive biochemical techniques unsustainable. Machine learning is suitable for such data-intensive task, but the presence of noise in protein datasets adds another level of difficulty. Hence, we propose a deep learning system based on a stacked denoising autoencoder that extracts robust features to improve predictive performance. We then feed the resulting features to a multilabel support-vector machine for classification. We evaluated on two protein benchmarks, and experimental results show that our system consistently produced the best performance against techniques that do not have a denoising or feature learning capability. This research demonstrates that learning robust representations from raw data can benefit the process of predicting protein functions.

Within-class imbalance problems often occur in imbalance classification which worsen the imbalance distribution problem and increase the learning concept complexity. However, most of existing methods for imbalanced classification focus on rectifying the between-class which are insufficiencies and inappropriateness in many different scenarios. This paper proposes a novel quasi-linear SVM with local offset adjustment method for imbalance classification problem. Our chief aim is to use leaning offsets of sub-clusters obtained according to imbalance ratios of sub-clusters to adjust classifier to achieve the best results. For this purpose, firstly, a geometry-based partitions method for imbalance dataset is introduced to partition the input space into several linearly separable partitions so as to construct a quasi-linear kernel and obtain an SVM classifier. Then a local offset method based on F-score value for linearly separable imbalance dataset is introduced to obtain leaning offset of each partition. At last the quasi-linear SVM with local offset adjustment is used to get the classifier for imbalance datasets. Simulation results on different real different real world datasets show that the proposed method is effective for imbalanced data classifications.

In this paper, aiming at the cumbersome solution of control law in neural network predictive control algorithm, a quasi-linear neural network identification and predictive control algorithm is proposed. The recurrent neural network is embedded into the quasi-linear model, which can be viewed as a quasi-ARX model macroscopically. In the quasi-linear recurrent neural network predictive control, the solution of the control law only need one-step derivation, which can greatly simplify the solution process of control law. At the same time, the quasi-linear recurrent neural network can effectively restrain the over-fitting problem in the identification process. Theoretical analysis and simulations are given to prove the simplicity and effectiveness of the proposed computing method.

Anomaly detection on system logs is to report system failures with utilization of console logs collected from devices, which ensures the reliability of systems. Most previous researches split logs into sequential time windows and regarded each window as an independent instance for classification using popular machine learning methods like support vector machine(SVM), however, neglected the time patterns under logs. Those approaches also suffer from information loss due to the vector representation, and high dimensionality if there is a large number of log events. To make up these deficiencies, unlike most traditional methods that used a vector to represent a period behavior at the macro level, we construct a 2D matrix to reveal more detailed system behaviors in the time period by dividing each window into sequential subwindows. To provide a more efficient representation, we further use the ant colony optimization algorithm to find a highly-coupled event template as the horizontal index of the 2D window matrix to replace the disordered one. To capture time dependencies, a multi-module convolutional auto-encoder is configured as that different paralleled modules scan among different time intervals to extract information respectively. These features are then concatenated in latent space as the final input, which contains diversified time information, for classification by SVM. The experiments on Blue Gene/L log dataset showed that our proposed method outperforms the state-of-art SVM method.

This paper proposes a novel method for one-class classification by using support vector machine (SVM) based on a divide-and-conquer strategy. An s% winner-take-all autoencoder is applied to realize a sophisticated partitioning which divides the dataset into many clusters. For each cluster, data points are separated from the origin in the feature space like a traditional one-class SVM (OCSVM). By designing a gated linear network, and generating the gate signal from the autoencoder, the proposed OCSVM is implemented in an exact same way as a standard OCSVM with a quasi-linear kernel composed by using a base kernel with the gate signals. Comparing to a traditional OCSVM, the proposed quasi-linear OCSVM is expected to capture a more compact region in the input space. The compact region will decrease the probability of outlier objects falling inside the domain of classifier, which give a better performance. The proposed quasi-linear OCSVM method is applied to different real-world datasets, and simulation results confirm the effectiveness of the proposed method.

A gated linear network is able to mimic the functionality of a pre-trained neural network with a compound activation function R(x) = x * S(x). An SVM can then be formulated to further implicitly optimize the gated linear network, in which a quasi-linear kernel is composed by using the gate signal S(x) generated from the pre-trained neural network. In this way, we realize a neural network based kernel learning. In this paper, a distance metric learning is applied to improving the kernel learning. In the pre-training of neural network, the loss function of distance metric learning is used as a regularization term. With the loss function of distance metric learning, the samples from within-class become closer and that from between-class become farther, which can improve the quasi-linear kernel. Accordingly, the classifier optimized by SVM with quasi-linear kernel will have better performance. The proposed classification method is applied to different real-world datasets and simulation results confirm the effectiveness of the proposed method.

Learning new representations from data has been effective in predicting protein functions. However, common techniques tend to extract features irrelevant to the classification task. We propose an autoencoder network that selectively extracts features to produce meaningful representations. By increasing the activation of neurons kept by a winner-take-all operation, hidden units compete to form a subset that encodes relevant features, a process dubbed as mutual competition. We test this method on protein benchmarks, evaluating feature score distribution and classification performance. Results show that the autoencoder extracted features relevant to the classification task, and significantly outperformed other techniques in literature based on non-parameteric statistical tests. This demonstrates that adding competition between neurons encodes meaningful features, further improving the prediction of protein functions.

Genetic Algorithms are commonly used to generate high-quality solutions to combinational optimization problems. However, the execution time can become a limiting factor for large and complex problems. In this paper, we propose a parallel Genetic Algorithm consisting of an island model at the upper level and a fine-grained model at the lower level. This design is highly consistent with the CUDA framework in order to get the maximum speedup without compromising to solutions' quality. As several parameters control the performance of the hybrid method, we test them by a flexible flow shop scheduling problem and analyze their influence. Finally, numerical experiments show that our approach cannot only obtain competitive results but also reduces execution time by setting a medium size selection diameter, a relatively large island size and a wide range size migration interval.

A mixture of multiple linear classifiers is famous for its efficiency and effectiveness to tackle nonlinear classification problems. Each classifier contains one linear function multiplied with a gated function, which restricts its corresponding classifier to a local region. Previous researches mainly focus on the partition of local regions, since its quality directly determines the performance of mixture models. However, in real-world data sets, imbalanced and insufficient labeled data are two frequently encountered problems, which also have large influences on the performance of learned classifiers but are seldom considered or explored in the context of mixture models. In this paper, these missing components are introduced into the original formulation of mixture models, namely, a sample weighting scheme for imbalanced data distributions and a manifold regularization to leverage unlabeled data. Then, two solutions with closed form are provided for parameter optimization. Experimental results in the end of our paper exhibit the significance of the added components. As a result, a mixture of multiple linear classifiers can be extended to imbalanced and semi-supervised learning problems.

A deep neural network (DNN) is called as a deep rectified network (DRN), if using Rectified Linear Units (ReLUs) as its activation function. In this paper, we show its parameters can be seen to play two important roles simultaneously: one for determining the subnetworks corresponding to the inputs and the other for the parameters of those subnetworks. This observation leads our paper to proposing a method to combine a DNN and an SVM, as a deep classifier. For a DRN trained by a common tuning algorithm, a multilayer gated bilinear classifier is designed to mimic its functionality. Its parameter set is duplicated into two independent sets, playing different roles. One set is used to generate gate signals so as to determine subnetworks corresponding to its inputs, and keeps fixed when optimizing the classifier. The other set serves as parameters of subnetworks, which are linear classifiers. Therefore, their parameters can be implicitly optimized by applying SVM optimizations. Since the DRN is only to generate gate signals, we show in experiments, that it can be trained by using supervised, or unsupervised learning, and even by transfer learning.

For many problems in machine learning fields, the data are nonlinearly distributed. One popular way to tackle this kind of data is training a local kernel machine or a mixture of several locally linear models. However, both of these approaches heavily relies on local information, such as neighbor relations of each data sample, to capture potential data distribution. In this paper, we show the non-local information is more efficient for data representation. With an implementation of a winner-take-all autoencoder, several non-local templates are trained to trace the data distribution and to represent each sample in different subspaces with a suitable weight. By training a linear model for each subspace in a divide and conquer manner, one single support vector machine can be formulated to solve nonlinear classification problems. Experimental results demonstrate that a mixture of multiple linear classifiers from non-local information performs better than or is at least competitive with state-of-the-art mixtures of locally linear models.

In this paper, a novel fast support vector machine (SVM) method combining with the deep quasi-linear kernel (DQLK) learning is proposed for large scale image classification. This method can train large-scale dataset with SVM fast using less memory space and less training time. Since SVM classifiers are constructed by support vectors (SVs) that lie close to the separation boundary, removing the other samples that are not relevant to SVs has no effect on building the separation boundary. In other word, we need to reserve the boundary samples that are likely to be SVs. The proposed method uses an approximate separation classifier obtained by training a small subset selected from training data randomly as a reference to detect and remove non-relevant samples whose normalized algebraic distance to the reference classification boundary is larger than a threshold. The proposed method is implemented in the feature space. Therefore, by means of a good kernel method the proposed method can train high dimension data and image data. The DQLK method is used to extract and construct kernel matrix for the proposed method. Experimental results on different datasets and expended very large scale datasets show that the proposed method obtains outstanding ability to deal with very large scale image classification.

Distance metric learning focuses on learning one global or multiple local distance functions to draw similar instances close to each other and push away dissimilar ones. Most existing work has to do matrix projection to learn distance functions. In this paper, we present a novel distance function learning model which is based on eigenvalue fine tuning. Our model not only is able to learn the global distance function but also can be easily adopted into local metric learning tasks. From the perspective of dimension reduction, the proposed model can measure how much information has been preserved after feature transformation. Moreover, we connect our model with principal components analysis to improve its performance by introducing the label information. Experimental results have demonstrated the effectiveness of the proposed method.

Automatic registration is an important task to prepare aligned 2D images for 3D structure visualization, and it is a challenging problem especially for the microscope images. This paper proposes a novel coevolution-based coarse-to-fine registration method, aiming to align the regions of interest (ROIs) in the image sequence. Firstly, a coarse registration for whole images is executed by a scale-invariant feature transform (SIFT) based method, which can facilitate the segmentation of ROIs. Secondly, a fine registration for the segmented ROIs is done by a genetic algorithm (GA) with a novel coevolutionary strategy. Experimental results demonstrate the good performance of the proposed method and it is also successfully applied to the renal biopsy image sequence.

In the multi-label classification issue, some implicit constraints and dependencies are always existed among labels. Exploring the correlation information among different labels is important for many applications. It not only can enhance the classifier performance but also can help to interpret the classification results for some specific applications. This paper presents an improved multi-label classification method based on local label constraints and classifier chains for solving multi-label tasks with large number of labels. Firstly, in order to exploit local label constraints in multi-label problem with large number of labels, clustering approach is utilized to segment training label set into several subsets. Secondly, for each label subset, local tree-structure constraints among different labels are mined based on mutual information metric. Thirdly, based on the mined local tree-structure label constraints, a variant of classifier chain strategy is implemented to enhance the multi-label learning system. Experiment results on five multi-label benchmark datasets show that the proposed method is a competitive approach for solving multi-label classification tasks with large number of labels.

The control system is a key technology to extract maximum energy from the incident wind. By regulating aerodynamic control, it is possible to adapt the changes in wind speed by controlling shaft speed. Thus, the turbine generator can track maximum power extracted from wind. In this paper, we propose a Lyapunov based switching control under quasi-linear ARX neural network (QARXNN) model to track maximum power of wind energy conversion system. The switching index is used to measure the stability of nonlinear controller and selects linear or nonlinear controller in order to ensure the stability. Interestingly, a simple switching law can be built utilizing the parameters of model directly. Finally, we have compared the proposed algorithm of switching controller with another algorithm. The results show that the proposed algorithm has better control performance.

In this paper, we introduce a data-dependent kernel called deep quasi-linear kernel, which can directly gain a profit from a pre-trained feedforward deep network. Firstly, a multi-layer gated bilinear classifier is formulated to mimic the functionality of a feed-forward neural network. The only difference between them is that the activation values of hidden units in the multi-layer gated bilinear classifier are dependent on a pre-trained neural network rather than a pre-defined activation function. Secondly, we demonstrate the equivalence between the multi-layer gated bilinear classifier and an SVM with a deep quasi-linear kernel. By deriving a kernel composition function, traditional optimization algorithms for a kernel SVM can be directly implemented to implicitly optimize the parameters of the multi-layer gated bilinear classifier. Experimental results on different data sets show that our proposed classifier obtains an ability to outperform both an SVM with a RBF kernel and the pre-trained feedforward deep network.

Kernel functions based machine learning algorithms have been extensively studied over the past decades with successful applications in a variety of real-world tasks. In this paper, we formulate a kernel level composition method to embed multiple local classifiers (kernels) into one kernel function, so as to obtain a more flexible data-dependent kernel. Since such composite kernels are composed by multiple local classifiers interpolated with several localizing gating functions, a specific learning process is also introduced in this paper to pre-determine their parameters. Experimental results are provided to validate two major perspectives of this paper. Firstly, the introduced learning process is effective to detect local information, which is essential for the parameter pre-determination of the localizing gating functions. Secondly, the proposed composite kernel has a capacity to improve classification performance.

This paper presents a new strategy to build multi tree hierarchical structure SVM which can get a more efficient and accuracy classification model for multiclass problems. Base on the theory of Binary Tree SVM (BTS), we proposed an improvement algorithm which extend binary tree structure to a multi tree structure, In the multi tree hierarchical structure, similarity clustering method was proposed to cluster classes to groups in each non-leaf node. In order to get a multi node division, one-against-all (OAA) was applied to train those groups rather than classes. The proposed method can avoid data imbalanced problem occurred in OAA, also the classification area of classifier in the upper layer is larger than classifier in lower layer. Compared with other several well-known methods, experiments on many data sets demonstrate that our method can reduce the number of classifiers in the testing phase and get a higher accuracy.

This paper proposes an improved SVM based multi-label classification method by using relationship among labels. Following a traditional multi-label solution, binary relevance (BR) method is first used to decompose the multi-label classification problem into multiple binary classification sub-problems, each of which is solved by an SVM classifier. By using Platt's sigmoid technique, each SVM classifier gives probability output for the following correction. A probability model is introduced to estimate the relationship among labels. The extracted label relationship is then applied to correct the outputs of SVM classifiers, in which a dynamic weight strategy is further introduced. Numerical experiments on widely used benchmark datasets show that the proposed method can improve the accuracy of multi-label classification when compared with traditional BR method and some other conventional multi-label classification methods.

This paper proposes a geometric way to construct a quasi-linear kernel by which a quasi-linear support vector machine (SVM) is performed. A quasi-linear SVM is a SVM with quasi-linear kernel, in which the nonlinear separation boundary is approximated by using multi-local linear boundaries with interpolation. However, the local linearity extraction for the composition of quasi-linear kernel is still an open problem. In this paper, according to the geometric theory, a method based on piecewise linear classifier is proposed to extract the local linearity in a more precise and efficient way. We firstly construct a function set including multiple linear functions and each of those functions reflects one part of linearity of the whole nonlinear separation boundary. Then the obtained local linearity is added as prior information into the composition of quasi-linear kernel. Experimental results on synthetic data sets and real world data sets show that our proposed method is effective to improve classification performances.

This paper presents a Transductive Support Vector Machine (TSVM) with quasi-linear kernel based on a clustering assumption for semi-supervised classification. Since the potential separating boundary is located in low density area between classes, a modified density clustering method by considering label information is firstly introduced to extract the information of potential separating boundary in low density region between different classes. Then the information is used to compose a quasi-linear kernel for the TSVM. The optimization of TSVM is further speeded up by developing a pairwise label switching method on minimal sets. Experiment results on benchmark datasets show that the proposed method is effective and improves classification performances.

Differential evolution (DE) is an effective and efficient evolutionary algorithm in continuous space. The setting of control parameters is highly relevant with the convergence efficiency, and varies with different optimization problems even at different stages of evolution. Self-adapting control parameters for finding global optima is a long-term target in evolutionary field. This paper proposes a two-layered DE (TLDE) with self-adaptive control parameters combined with niching method based mutation strategy. The TLDE consists of two DE layers: a bottom DE layer for the basic evolution procedure, and a top DE layer for control parameter adaptation. Both layers follow the procedure of DE. Moreover, to mitigate the common phenomenon of premature convergence in DE, a clearing niching method is brought out in finding efficient mutation individuals to maintain diversity during the evolution and stabilize the evolution system. The performance is validated by a comprehensive set of twenty benchmark functions in parameter optimization and competitive results are presented.

A quasi-ARX (quasi-linear ARX) neural network (QARXNN) model is able to demonstrate its ability for identification and prediction highly nonlinear system. The model is simplified by a linear correlation between the input vector and its nonlinear coefficients. The coefficients are used to parameterize the input vector performed by an embedded system called as state dependent parameter estimation (SDPE), which is executed by multi layer parceptron neural network (MLPNN). SDPE consists of the linear and nonlinear parts. The controller law is derived via SDPE of the linear and nonlinear parts through switching mechanism. The dynamic tracking controller error is derived then the stability analysis of the closed-loop controller is performed based Lyapunov theorem. Linear based adaptive robust control and nonlinear based adaptive robust control is performed with the switching of the linear and nonlinear parts parameters based Lyapunov theorem to guarantee bounded and convergence error.

Clustering, one of the important data mining techniques, has two main processing methods on data-based similarity clustering and space-based density grid clustering. The latter has more advantage than the former on larger and multiple shape and density dataset. However, due to a global partition of existing grid-based methods, they will perform worse when there is a big difference on the density of clusters. In this paper, we propose a novel algorithm that can produces appropriate grid space in different density regions by simulating cell division process. The time complexity of the algorithm is O(n) in which n is number of points in dataset. The proposed algorithm will be applied on popular chameleon datasets and our synthetic datasets with big density difference. The results show our algorithm is effective on any multi-density situation and has scalability on space optimization problems.

Support Vector Data Description (SVDD) inherits properties of Support Vector Machines (SVM) and has become a prominent One Class Classifier (OCC). Same to standard SVM, its O (n3) time and O (n2) space complexities, where n is the number of training samples, have become major limitations in cases of large training datasets. As a simple and effective method, reducing the size of training dataset through reserving only samples mostly relevant to learned classifier, can be adopted to overcome the limitations. A trained SVDD enclosed decision boundary always locates on edge area of data distribution and is decided by a small subset of Support Vectors(SVs). Therefore, in this paper, we present a method based on edge detection such that edge samples mostly relevant to decision boundary can be preserved. And clustering techniques are also be applied to keep centroids representing the global distribution properties so as to avoid over-outside of decision boundary. To restrict the influences of noises, each training pattern is assigned with a weight. Experiments on real and artificial data sets prove that the classifier trained on reconstruction training set consisting of edge points and centroids can preserve performance with much faster training speed.

This paper proposes a self-organizing maps (SOM) based kernel composition method for the quasi-linear support vector machine (SVM). The quasi-linear SVM is SVM model with quasi-linear kernel, in which the nonlinear separation hyperplane is approximated by multiple local linear models with interpolation. The basic idea underlying the proposed method is to use clustering and projection properties of SOM to partition the input space and construct a SOM based quasi-linear kernel. By effectively extracting the distribution information using SOM, the quasi-linear SVM with the SOM-based quasi-linear kernel is expected to have better performance in the cases of high-noise and high-dimension. Experiment results on synthetic datasets and real world datasets show the effectiveness of the proposed method.

This paper focuses on semi-supervised classification problem by using Transductive Support Vector Machine. Traditional TSVM for semi-supervised classification firstly train an SVM model with labeled data. Then use the model to predict unlabeled data and optimize unlabeled data prediction to retrain the SVM. TSVM always uses a predefined kernel and fixed parameters during the optimization procedure and they also suffers potential over-fitting problem. In this paper we introduce proposed quasi-linear kernel to the TSVM. An SVM with quasi-linear kernel realizes an approximate nonlinear separation boundary by multi-local linear boundaries with interpolation. By applying quasi-linear kernel to semi-supervised classification it can avoid potential over-fitting and provide more accurate unlabeled data prediction. After unlabeled data prediction optimization, the quasi-linear kernel can be further adjusted considering the potential boundary data distribution as prior knowledge. We also introduce a minimal set method for optimizing unlabeled data prediction. The minimal set method follows the clustering assumption of semi-supervised learning. The pairwise label switching is allowed between minimal sets. It can speed up optimization procedure and reduce influence from label constrain in TSVM. Experiment results on benchmark gene datasets show that the proposed method is effective and improves classification performances.

Multi-label classification is an extension of traditional classification problem in which each instance is associated with a set of labels. For some multi-label classification tasks, labels are usually overlapped and correlated, and some implicit constraint rules are existed among the labels. This paper presents an improved multi-label classification method based on label ranking strategy and label constraints. Firstly, one-against-all decomposition technique is used to divide a multilabel classification task into multiple independent binary classification sub-problems. One binary SVM classifier is trained for each label. Secondly, based on training data, label constraint rules are mined by association rule learning method. Thirdly, a correction model based on label constraints is used to correct the probabilistic outputs of SVM classifiers for label ranking. Experiment results on three well-known multi-label benchmark datasets show that the proposed method outperforms some conventional multi-label classification methods.

This work exploits the idea on how to search parameter estimation and increase its convergence speed for the Liner Time Invariant (LTI) system. The convergence speed of parameter estimation is the one problem and plays an important role in the adaptive controller to increase performance. The well-known algorithm is the recursive least square algorithm. However, the speed of convergence is still low and is influenced by the number of sampling, which is represented by the limited availability for the information vector. We offer a new method to increase the convergence speed by applying Quasi-ARX model. Quasi-ARX model performs two steps identification process by presenting parameter estimation as a function over time. The first, parameters estimation of macro-part sub-model are searched by the least square error, and the second is to sharpen the searching by performing backpropagation learning of multi layer parceptron network.

In this paper, an SVM-based approach is proposed for stock market trend prediction. The proposed approach consists of two parts: feature selection and prediction model. In the feature selection part, a correlation-based SVM filter is applied to rank and select a good subset of financial indexes. And the stock indicators are evaluated based on the ranking. In the prediction model part, a so called quasi-linear SVM is applied to predict stock market movement direction in term of historical data series by using the selected subset of financial indexes as the weighted inputs. The quasi-linear SVM is an SVM with a composite quasi-linear kernel function, which approximates a nonlinear separating boundary by multi-local linear classifiers with interpolation. Experimental results on Taiwan stock market datasets demonstrate that the proposed SVM-based stock market trend prediction method produces better generalization performance over the conventional methods in terms of the hit ratio. Moreover, the experimental results also show that the proposed SVM-based stock market trend prediction system can find out a good subset and evaluate stock indicators which provide useful information for investors.

This paper focuses on imbalanced dataset classification problem by using SVM and oversampling method. Traditional oversampling method increases the occurrence of over-lapping between classes, which leads to poor generalization of SVM classification. To solve this problem this paper proposes a combined method of quasi-linear SVM and assembled SMOTE. The quasi-linear SVM is an SVM with quasi-linear kernel function. It realizes an approximate nonlinear separation boundary by mulit-local linear boundaries with interpolation. The assembled SMOTE implements oversampling with considering of the data distribution information and avoids occurrence of overlapping between classes. Firstly, a partition method based on Minimal Spanning Tree is proposed to obtain local linear partitions, each of which can be separated with one linear separation boundary. Secondly, using the information of local linear partitions, the assembled SMOTE generates synthetic minority class samples. Finally, the quasi-linear SVM realizes a classification of oversampled datasets in the same way as a standard SVM by using a composite quasi-linear kernel function. Experiment results on artificial data and benchmark datasets show that the proposed method is effective and improves classification performances.

This paper proposes a hierarchical multi-label classification method based on SVM with composite kernel for solving gene function prediction. The hierarchical multi-label classification problem is resolved into a set of binary classification tasks. A composite kernel based SVM (ck-SVM) is introduced to deal with the binary classification tasks. In estimation procedure of ck-SVM, a supervised clustering with over-sampling strategy is introduced for solving imbalance dataset learning problem and improve classification performance. Experimental results on benchmark datasets demonstrate that the proposed method improves the classification performance efficiently.

In recent years, support vector regression (SVR) has attracted much attention for nonlinear system identification. It can solve nonlinear problems in the form of linear expressions within the linearly transformed space. Commonly, the convenient kernel trick is applied, which leads to implicit nonlinear mapping by replacing the inner product with a positive definite kernel function. However, only a limited number of kernel functions have been found to work well for the real applications. Moreover, it has been pointed that the implicit nonlinear kernel mapping is not always good, since it may faces the potential over-fitting for some complex and noised learning task. In this paper, explicit nonlinear mapping is learnt by means of the quasi-ARX modeling, and the associated inner product kernel, which is named quasi-linear kernel, is formulated with nonlinearity tunable between the linear and nonlinear kernel functions. Numerical and real systems are simulated to show effectiveness of the quasi-linear kernel, and the proposed identification method is also applied to microarray missing value imputation problem.

This paper presents a method for nonlinear discriminant analysis utilizing a composite kernel which is derived from a combination of local linear models with interpolation. The underlying idea is to decompose a complex nonlinear problem into a set of simpler local linear problems. Combining with the theory of nonlinear classification based on kernels, the local linear models with interpolation can be formulated as a composite kernel based discriminant analysis form. In face recognition, linear discriminant analysis (LDA) has been widely adopted owing to its efficiency, but it fails to solve nonlinear problems. Conventional kernel based approaches such as generalized discriminant analysis (GDA) has been successfully applied to extend LDA to nonlinear pattern recognition tasks. However, selecting an appropriate kernel function is usually difficult. Utilizing an implicit kernel mapping may face potential over-training problems for some complex and noised tasks. Our proposed method gives an alternative solution for nonlinear discriminant analysis while the conventional linear and nonlinear approaches are difficult to achieve a satisfactory results. Experiments on both synthetic data and face data set show the effectiveness of the proposed methods.

In this note, we present the modeling of nonlinear dynamical systems with Quasi-ARX neural network using Lyapunov algorithm in learning process. This work exploits the idea on learning algorithm in nonlinear kernel part of Quasi-ARX model to improve stability and fast convergence of error. The proposed algorithm is then employed to model and predict a classical nonlinear system with input dead zone and nonlinear dynamic systems, exhibiting the effectiveness of proposed algorithm. Based on the result of simulation, the proposed algorithm can make the error in process learning become fast convergence, ultimately bounded, and the error distributed uniformly.

Quasi-ARX neurofuzzy (Q-ARX-NF) models have shown great approximation ability and usefulness in nonlinear system identification and control. However, the incorporated neurofuzzy networks suffer from the curse-of-dimensionality problem, which may result in high computational complexity and over-fitting. In this paper, support vector regressor (SVR) based identification approach is used to reduce computational complexity with the help of transforming the original problem into Lagrange space, which is only sensitive to the number of data samples. Furthermore, to improve the generalization capability, a parsimonious model structure is obtained by eliminating insignificant input variables for the incorporated neurofuzzy network, which is implemented by genetic algorithm (GA) based input selection method with a novel fitness evaluation function. Two numerical simulations are tested to show the effectiveness of the proposed method.

Sequencing by hybridization is a promising cost-effective technology for high-throughput DNA sequencing via microarray chips. However, due to the effects of spectrum errors rooted in experimental conditions, an accurate and fast reconstruction of original sequences has become a challenging problem. In the last decade, a variety of analyses and designs have been tried to overcome this problem, where different strategies have different trade-offs in speed and accuracy. Motivated by the idea that the errors could be identified by analyzing the interrelation of spectrum elements, this paper presents a constructive heuristic algorithm, featuring an accurate reconstruction guided by a set of well-defined criteria and rules. Instead of directly reconstructing the original sequence, the new algorithm first builds several accurate short fragments, which are then carefully assembled into a whole sequence. The experiments on benchmark instance sets demonstrate that the proposed method can reconstruct long DNA sequences with higher accuracy than current approaches in the literature.

Sequencing by hybridization is a promising cost-effective technology for high-throughput DNA sequencing via microarray chips. However, due to the effects of spectrum errors rooted from experimental conditions, a fast and accurate reconstruction of original sequences has become a challenging problem. In the last decade, a variety of analyses and designs have been tried to overcome this problem, where different strategies have different tradeoffs in speed and accuracy. Motivated by the idea that the errors could be identified by analyzing the interrelation of spectrum elements, this paper presents a new constructive heuristic algorithm, featuring an accurate reconstruction guided by a set of well-defined criteria and rules. The experiments on benchmark instance sets demonstrate that the proposed method can reconstruct long DNA sequences more accurately than current approaches in the literature.

Estimation of Distribution Algorithms (EDAs) still suffer from the drawback of premature convergence for solving the optimization problems with irregular and complex multimodal landscapes. In this paper, we propose an adaptive niching EDA based on Affinity Propagation (AP) clustering analysis. The AP clustering is used to adaptively partition the niches and mine searching information from the evolution process. The obtained information is successfully utilized to improve the EDA performance by a balance niching searching strategy. Two different categories of optimization problems are used to evaluate the proposed adaptive niching EDA. The first is the continuous EDA based on single Gaussian probabilistic model to solve two benchmark functional multimodal optimization problems. The second is a real complicated discrete EDA optimization problem, the protein 3-D HP model based on k-order Markov probabilistic model. The experiment studies demonstrate that the proposed adaptive niching EDA is an efficient method.

In this paper, we propose a hierarchical frequency sensitive competitive learning (HFSCL) method to achieve color quantization (CQ). In HFSCL, the appropriate number of quantized colors and the palette can be obtained by an adaptive procedure following a binary tree structure with nodes and layers. Starting from the root node that contains all colors in an image until all nodes are examined by split conditions, a binary tree will be generated. In each node of the tree, a frequency sensitive competitive learning (FSCL) network is used to achieve two-way division. To avoid over-split, merging condition is defined to merge the clusters that are close enough to each other at each layer. Experimental results show that HFSCL has the desired ability for CQ.

In this paper, an improved multi-label classification is proposed based on label ranking and delicate decision boundary SVM. Firstly, an improved probabilistic SVM with delicate decision boundary is used as the scoring method to obtain a proper label rank. It can improve the probabilistic label rank by introducing the information of overlapped training samples into learning procedure. Secondly, a threshold selection related with input instance and label rank is proposed to decide the classification results. It can estimate an appropriate threshold for each testing instance according to the characteristics of instance and label rank. Experimental results on four multi-label benchmark datasets show that the proposed method improves the performance of classification efficiently, compared with binary SVM method and some existing well-known methods.

Brain-Computer Interface (BCI) is a system provides an alternative communication and control channel between the human brain and computer. In Motor Imagery-based (MI) BCI system, Common Spatial Pattern (CSP) is frequently used for extracting discriminative patterns from the electroencephalogram (EEG). There are many studies have proven that the performance of CSP has a very important relation with the choice of operational frequency band. As the fact that the CSP features at different frequency bands contain discriminative and complementary information for classification, this paper proposes a new frequency band selection method to find the best frequency band set on which subject-specifics CSP are complementary for MI classification. Compared to the performance offered by the existing method based on frequency band partition, the proposed algorithm can yield error rate reductions of 49.70% for the same BCI competition dataset.

This paper presents a novel approach for designing adaptive controller of nonlinear dynamical systems based on an improved quasi-ARX neural network prediction model. The improved quasi-ARX neural network prediction model has two parts: the linear part is used for stability and the nonlinear part is used to satisfy accuracy requirement. Then, we can obtain a linear controller and a nonlinear controller based on the characteristic of the improved quasi-ARX neural network prediction model. A fuzzy switching algorithm is designed between the two controllers. Theory analysis and simulations are given to show the effectiveness of the proposed method both on stability and accuracy.

This paper proposes an improved Hierarchical Multi-label Classification (HMC) method for solving the gene function prediction. The HMC task is transferred into a series of binary SVM classification tasks. By introducing the hierarchy constraint into learning procedures, two measures with incorporating prior information are implemented to improve the HMC performance. Firstly, for imbalanced functional classes, a hierarchical SMOTE is proposed as over-sampling preprocessing to improve the SVM learning performance. Secondly, an improved True Path Rule consistency approach is introduced to ensemble the results of binary probabilistic SVM classifications. It can improve the classification results and guarantee the hierarchy constraint of classes.

This paper presents a novel image edge detection method based on a simplified pulse coupled neural network with anisotropic interconnections (PCNNAI) by applying an anisotropic linking mechanism. PCNNAI utilizes the anisotropic linking mechanism to create an adaptive synaptic weight matrix to achieve the anisotropic interconnection model among neurons. Therefore, the neurons corresponding to edge and non-edge pixels will receive different feedback signal from neighborhood. Due to the PCNN structure the edges will be detected by different internal activity of edge neurons and non-edge neurons. Comparing with conventional PCNN edge detection methods, PCNNAI simplifies the system structure and the outputs are controllable, meanwhile PCNNAI also achieves more accurate results than the classical image edge detectors. Experimental results show that PCNNAI is effective at image edge detection.

Multi-label classification is an extension of traditional classification problem in which each instance is associated with a set of labels. Recent research has shown that the ranking approach is an effective way to solve this problem. In the multi-labeled sets, classes are often related to each other. Some implicit constraint rules are existed among the labels. So we present a novel multi-label ranking algorithm inspired by the pairwise constraint rules mined from the training set to enhance the existing method. In this method, one-against-all decomposition technique is used firstly to divide a multi-label problem into binary class sub-problems. A rank list is generated by combining the probabilistic outputs of each binary Support Vector Machine (SVM) classifier. Label constraint rules are learned by minimizing the ranking loss. Experimental performance evaluation on well-known multi-label benchmark datasets show that our method improves the classification accuracy efficiently, compared with some existed methods.

The protein structure prediction (PSP) problem is one of the most important problems in computational biology. This paper proposes a novel Estimation of Distribution Algorithms (EDAs) based method to solve the PSP problem on HP model. Firstly, a composite fitness function containing the information of folding structure core formation is introduced to replace the traditional fitness function of HP model. It can help to select more optimum individuals for probabilistic model of EDAs algorithm. And a set of guided operators are used to increase the diversity of population and the likelihood of escaping from local optima. Secondly, an improved backtracking repairing algorithm is proposed to repair invalid individuals sampled by the probabilistic model of EDAs for the long sequence protein instances. A detection procedure of feasibility is added to avoid entering invalid closed areas when selecting directions for the residues. Thus, it can significant reduce the number of backtracking operation and the computational cost for long sequence protein. Experimental results demonstrate that the proposed method outperform the basic EDAs method. At the same time, it is very competitive with the other existing algorithms for the PSP problem on lattice HP models.

Glomerulus is the filtering unit of the kidney. In the computer aided diagnosis system designed for kidney disease, glomerulus extraction is an important step for analyzing kidney-tissue image. Against the disadvantages of traditional methods, this paper proposes a glomerulus extraction method using genetic algorithm for edge patching. Firstly, Canny edge detector is applied to get discontinuous edges of glomerulus. After labeling to remove the noises, genetic algorithm is used to search for optimal patching segments to join those edges together. Lastly, the edges and the patching segments with high fitness would be able to form the whole edge of the glomerulus. Experiments and comparisons indicate the proposed method can extract the glomerulus from kidney-tissue image both fast and accurately.

In a standard support vector machine (SVM), the training process has O(n3) time and O(n2) space complexities, where n is the size of training dataset. Thus, it is computationally infeasible for very large datasets. Reducing the size of training dataset is naturally considered to solve this problem. SVM classifiers depend on only support vectors (SVs) that lie close to the separation boundary. Therefore, we need to reserve the samples that are likely to be SVs. In this paper, we propose a method based on the edge detection technique to detect these samples. To preserve the entire distribution properties, we also use a clustering algorithm such as K-means to calculate the centroids of clusters. The samples selected by edge detector and the centroids of clusters are used to reconstruct the training dataset. The reconstructed training dataset with a smaller size makes the training process much faster, but without degrading the classification accuracies.

In this paper, an automatic segmentation method based on self-organizing feature map (SOFM) neural network (NN) is presented for color images. First, a binary tree clustering procedure is used to cluster the colors in an image. In each node of the tree, a SOFM NN is used as a classifier which is fed by image color values. The output neurons of the SOFM NN define the color classes for each node. In our method, the number of color classes for each node is two. For each node of the tree, Hotelling transform based splitting condition is used to define if the current color classes should be split. To speed up the entire algorithm, a nearest neighbor interpolation is used to get the small training set for SOFM NN. Once the colors in an image are clustered, it is easy to segment a target by analyzing the color feature in an image. The method is independent of the color scheme, so it is applicable to any type of color images. Our experimental results show the validity of the proposed method.

Deceptive problems are a class of challenging problems for conventional genetic algorithms (GAs), which usually mislead the search to some local optima rather than the global optimum. This paper presents an improved genetic algorithm with reserve selection to solve deceptive problems. The concept ldquopotentialrdquo of individuals is introduced as a new criterion for selecting individuals for reproduction, where some individuals with low fitness are also let survive only if they have high potentials. An operator called adaptation is further employed to release the potentials for approaching the global optimum. Case studies are done in two deceptive problems, demonstrating the effectiveness of the proposed algorithm.

An efficient algorithm for mining important association rule from multi-relational database using distributed mining ideas. Most existing data mining approaches look for rules in a single data table. However, most databases are multi-relational. In this paper, we present a novel distributed data-mining method to mine important rules in multiple tables (relations) and combine the method with genetic algorithm to enhance the mining efficiency. Genetic algorithm is in charge of finding antecedent rules and aggregate of transaction set that produces the corresponding rule from the chief attributes. Apriori and statistic method is in charge of mining consequent rules from the rest relational attributes of other tables according to the corresponding transaction set producing the antecedent rule in a distributed way. Our method has several advantages over most exiting data mining approaches. First, it can process multi-relational database efficiently. Second, rules produced have finer pattern. Finally, we adopt a new concept of extended association rules that contain more import and underlying information.

This paper presents a novel support vector regression (SVR) network for financial time series prediction. The SVR network consists of two layers of SVR: transformation layer and prediction layer. The SVRs in the transformation layer forms a modular network; but distinguished with conventional modular networks, the partition of the SVR modular network is based on the output domain that has much smaller dimension. Then the transformed outputs from the transformation layer are used as the inputs for the SVR in prediction layer. The whole SVR network gives an online prediction of financial time series. Simulation results on the prediction of currency exchange rate between US dollar and Japanese Yen show the feasibility and the effectiveness of the proposed method.

Elevator group supervisory control systems (EGSCSs) are designed so that the movement of several elevators in a building is controlled efficiently. The efficient control of EGSCSs using conventional control methods is very difficult due to its complexity, so it is becoming popular to introduce artificial intelligence (AI) technologies into EGSCSs in recent years. As a new approach, a graph-based evolutionary method named genetic network programming (GNP) has been applied to the EGSCSs, and its effectiveness is clarified. The GNP can introduce various a priori knowledge of the EGSCSs in its node functions easily, and can execute an efficient rule-based group supervisory control that is optimized in an evolutionary way. Meanwhile, double-deck elevator systems (DDESs) where two cages are connected in a shaft have been developed for the rising demand of more efficient transport of passengers in high-rise buildings. The DDESs have specific features due to the connection of cages and the need for comfortable riding; so its group supervisory control becomes more complex and requires more efficient group control systems than the conventional single-deck elevator systems (SDESs). In this paper, a new group supervisory control system for DDESs using GNP is proposed, and its optimization and performance evaluation are done through simulations. First, optimization of the GNP for DDSEs is executed. Second, the performance of the proposed method is evaluated by comparison with conventional methods, and the obtained control rules in GNP are studied. Finally, the reduction of space requirements compared with SDESs is confirmed.

Upstream regions in the DNA sequence are characterized by the presence of short regulatory motifs, which function as target binding sites for transcription factors. Finding two genes with common motifs in their regulatory regions may aid users in identifying co-regulated genes or inferring regulatory modules. By modelling pattern occurrences in the regulatory regions with Poisson statistics, this paper presents a log likelihood ratio statistics-based distance measure to calculate pair-wise similarities between sequences. To perform cluster analysis of regulatory sequences, this paper introduces two clustering algorithms on the basis of the incorporation of the log likelihood ratio statistics-based distance into hierarchical clustering and Self-Organizing Map. The proposed approach has been tested on a synthetic dataset and a real biological example. The results indicate that, in comparison to traditional distance functions, the log likelihood ratio statistics-based similarity measure offers considerable improvements in the process of regulatory sequence-based gene classification.

This paper provides a deep insight into the performance of genetic algorithms with reserve selection (GARS), and investigates how parameters can be regulated to solve optimization problems more efficiently. First of all, we briefly present GARS, an improved genetic algorithm with a reserve selection mechanism which helps to avoid premature convergence. The comparable results to state-of-the-art techniques such as fitness scaling and sharing demonstrate both the effectiveness and the robustness of GARS in global optimization. Next, two strategies named static RS and dynamic RS are proposed for tuning the parameter reserve size to optimize the performance of GARS. Empirical studies conducted in several cases indicate that the optimal reserve size is problem dependent.

In this paper, trading rules on stock market using the genetic network programming (GNP) with Sarsa learning is described. GNP is an evolutionary computation, which represents its solutions using graph structures and has some useful features inherently. It has been clarified that GNP works well especially in dynamic environments since GNP can create quite compact programs and has an implicit memory function. In this paper, GNP is applied to creating a stock trading model. There are three important points: The first important point is to combine GNP with Sarsa learning which is one of the reinforcement learning algorithms. Evolution-based methods evolve their programs after task execution because they must calculate fitness values, while reinforcement learning can change programs during task execution, therefore the programs can be created efficiently. The second important point is that GNP uses candlestick chart and selects appropriate technical indices to judge the timing of the buying and selling stocks. The third important point is that sub-nodes are used in each node to determine appropriate actions (buying/selling) and to select appropriate stock price information depending on the situation. In the simulations, the trading model is trained using the stock prices of 16 brands in 2001, 2002 and 2003. Then the generalization ability is tested using the stock prices in 2004. From the simulation results, it is clarified that the trading rules of the proposed method obtain much higher profits than Buy&Hold method and its effectiveness has been confirmed.

Among several methods of extracting association rules that have been reported, a new evolutionary computation method named Genetic Network Programming (GNP) has also shown its effectiveness for small datasets that have a relatively small number of attributes. The aim of this paper is to propose a new method to extract association rules from large and dense datasets with a huge amount of attributes using GNP It consists of two level of processing. Server Level where conventional GNP based mining method runs in parallel and Client Level where files are considered as individuals and genetic operations are carried out over them. The algorithm starts dividing the large dataset into small datasets with appropiate size, and then each of them are dealt with GNP in parallel processing. The new association rules obtained in each generation are stored in a general global pool. We compared several genetic operators applied to the individuals in the Global Level. The proposed method showed remarkable improvements on simulations.

Genetic network programming (GNP) is an evolutionary computation which represents its solutions using graph structures. Since GNP can create quite compact programs and has an implicit memory function, it has been clarified that GNP works well especially in dynamic environments. In this paper, GNP is applied to creating a stock trading model. The first important point is to combine GNP with Actor-Critic which is one of the reinforcement learning algorithms. Evolution-based methods evolve their programs after task execution because they must calculate fitness values, while reinforcement learning can change programs during task execution, therefore the programs can be created efficiently. The second important point is that GNP with Actor-Critic (GNP-AC) can select appropriate technical indexes to judge the buying and selling timing of stocks using Importance Index especially designed for stock trading decision making. In the simulations, the trading model is trained using the stock prices of 20 brands in 2001, 2002 and 2003. Then the generalization ability is tested using the stock prices in 2004. From the simulation results, it is clarified that the trading rules of GNP-AC obtain higher profits than Buy&Hold method.

This paper applies a Adaptive Random search with Intensification and Diversification combined with Genetic Algorithm (RasID-GA) to neural network training. In the previous work, we proposed RasID-GA which combines the best properties of RasID and Genetic Algorithm for optimization. Neural networks are widely used in pattern recognition, system modeling, prediction and other areas. Although most neural network training uses gradient based schemes such as wellknown back-propagation (BP), but sometimes BP is easily dropped into local minima. In this paper, we train multi-branch neural networks using RasID-GA with constraint coefficient C by which the feasible solution space is controlled. In addition, we use Mackey-Glass time prediction to test a generalization ability of the proposed method.

Most association rule mining algorithms make use of discretization algorithms for handling continuous attributes. Discretization is a process of transforming a continuous attribute value into a finite number of intervals and assigning each interval to a discrete numerical value. However, by means of methods of discretization, it is difficult to get highest attribute interdependency and at the same time to get lowest number of intervals. In this paper we present an association rule mining algorithm that is suited for continuous valued attributes commonly found in scientific and statistical databases. We propose a method using a new graph-based evolutionary algorithm named “Genetic Network Programming (GNP)” that can deal with continues values directly, that is, without using any discretization method as a preprocessing step. GNP represents its individuals using graph structures and evolve them in order to find a solution; this feature contributes to creating quite compact programs and implicitly memorizing past action sequences. In the proposed method using GNP, the significance of the extracted association rule is measured by the use of the chi-squared test and only important association rules are stored in a pool all together through generations. Results of experiments conducted on a real life database suggest that the proposed method provides an effective technique for handling continuous attributes.

In this paper, we propose a Genetic Network Programming based method to mine equalized association rules in multi concept layers of ontology. We first introduce ontology to facilitate building the multi concept layers and propose Dynamic Threshold Approach (DTA) to equalize the different layers. We make use of an evolutionary computation method called Genetic Network Programming (GNP) to mine the rules and develop a new genetic operator to speed up searching the rule space. The simulation results show that our method could efficiently find some rules even in the early generations.

Recently, Artificial Intelligence (AI) technology has been applied to many applications. As an extension of Genetic Algorithm (GA) and Genetic Programming (GP), Genetic Network Programming (GNP) has been proposed, whose gene is constructed by directed graphs. GNP can perform a global searching, but its evolving speed is not so high and its optimal solution is hard to obtain in some cases because of the lack of the exploitation ability of it. To alleviate this difficulty, we developed a hybrid algorithm that combines Genetic Network Programming (GNP) with Ant Colony Optimization (ACO). Our goal is to introduce more exploitation mechanism into GNP. In this paper, we applied the proposed hybrid algorithm to a complicated real world problem, that is, Elevator Group Supervisory Control System (EGSCS). The simulation results showed the effectiveness of the proposed algorithm.

In order to increase the transportation capability of elevator group systems in high-rise buildings without adding elevator installation space, double-deck elevator system (DDES) is developed as one of the next generation elevator group systems. Artificial intelligence (AI) technologies have been employed to find some efficient solutions in the elevator group control systems during the late 20th century. Genetic Network Programming (GNP), a new evolutionary computation method, is reported to be employed as the elevator group system controller in some studies of recent years. Moreover, reinforcement learning (RL) is also verified to be useful for more improvements of elevator group performances when it is combined with GNP. In this paper, we proposed a new approach of DDES using GNP with RL, and did some experiments on a simulated elevator group system of a typical office building to check its efficiency. Simulation results show that the DDES using GNP with RL performs better than the one without RL in regular and down-peak time, while both of them outperforms a conventional approach and a heuristic approach in all three traffic patterns.

This paper investigates how genetic algorithms (GAs) can be improved to solve large-scale and complex problems more efficiently. First of all, we review premature convergence, one of the challenges confronted with when applying GAs to real-world problems. Next, some of the methods now available to prevent premature convergence and their intrinsic defects are discussed. A qualitative analysis is then done on the cause of premature convergence that is the loss of building blocks hosted in less-fit individuals during the course of evolution. Thus, we propose a new improver - GAs with Reserve Selection (GARS), where a reserved area is set up to save potential building blocks and a selection mechanism based on individual uniqueness is employed to activate the potentials. Finally, case studies are done in a few standard problems well known in the literature, where the experimental results demonstrate the effectiveness and robustness of GARS in suppressing premature convergence, and also an enhancement is found in global optimization capacity.

In this paper, the Genetic Network Programming (GNP) for creating trading rules on stocks is described. GNP is an evolutionary computation, which represents its solutions using graph structures and has some useful features inherently. It has been clarified that GNP works well especially in dynamic environments since GNP can create quite compact programs and has an implicit memory function. In this paper, GNP is applied to creating a stock trading model. There are three important points: The first important point is to combine GNP with Sarsa Learning which is one of the reinforcement learning algorithms. Evolution-based methods evolve their programs after task execution because they must calculate fitness values, while reinforcement learning can change programs during task execution, therefore the programs can be created efficiently. The second important point is that GNP uses candlestick chart and selects appropriate technical indices to judge the buying and selling timing of stocks. The third important point is that sub-nodes are used in each node to determine appropriate actions (buying/selling) and to select appropriate stock price information depending on the situation. In the simulations, the trading model is trained using the stock prices of 16 brands in 2001, 2002 and 2003. Then the generalization ability is tested using the stock prices in 2004. From the simulation results, it is clarified that the trading rules of the proposed method obtain much higher profits than Buy&Hold method and its effectiveness has been confirmed.

Several methods of extracting association rules have been reported. A new evolutionary computation method named Genetic Network Programming (GNP) has also been developed recently and its efectiveness is shown for small datasets. However, it has not been tested for large datasets, particularly in datasets with a large number of attributes. The aim of this paper is to extract association rules from large and dense datasets using GNP considering a real world database with a huge number of attributes. We propose a new method where a large database is divided into many small datasets, then each GNP deals with one dataset having attributes with appropiate size, which was selected randomly from a large dataset and generated genetically. These GNPs are processed in parallel. We then propose some new genetic operations to improve the number of rules extracted and their quality as well. The proposed method improves remarkably on simulations.

No abstract available.

Most association rule mining algorithms make use of discretization algorithms for handling continuous attributes. However, by means of methods of discretization, it is difficult to get highest attribute interdependency and at the same time to get lowest number of intervals. We propose a method using a new graph-based evolutionary algorithm named Network Programming (GNP) that can deal with continues values directly, that is, without using any discretization method as a preprocessing step. GNP is one of the evolutionary optimization techniques, which uses directed graph structures as solutions and is composed of three kinds of nodes: start node, judgment node and processing node. Once GNP is booted up, firstly the execution starts from the start node, secondly the next node to be executed is determined according to the judgment and connection from the current activated node. The features of GNP are described as follows. First, it is possible to reuse nodes; because of this, the structure is compact. Second, GNP can find solutions of problems without bloat, which can be sometimes found in Genetic Programming (GP), because of the fixed number of nodes in GNP. Third, nodes that are not used at the current program executions will be used for future evolution. Fourth, GNP is able to cope with partially observable Markov processes. In this paper, we propose a method that can deal with continuous attributes, where attributes in databases correspond to judgment nodes in GNP and each continuous attribute is checked whether its value is greater than a threshold value and the association rules are represented as the connections of the judgment nodes. Threshold ai is firstly determined by calculating the mean µi and standard deviation si of all attribute values of Ai. Then, initial threshold ai is selected randomly between the interval [µi - aisi, µi + aisi] where ai is a parameter to determine the range of the interval. Once the threshold ai is selected for all attributes, each value of the attribute Ai is checked if it is greater than the threshold ai in the judgment nodes of the proposed method. In addition to that, the threshold ai is also evolved by mutation between [µi - aisi, µi + aisi] in every generation in order to obtain as many association rules as possible. The features of the proposed method are as follows compared with other methods: 1) Extracts rules without identifying frequent itemsets used in Apriori-like mining methods. 2) Stores extracted important association rules in a pool all together through generations. 3) Measures the significance of associations via the chi-squared test. 4) Extracts important rules sufficient enough for user's purpose in a short time. 5) The pool is updated in every generation and only important association rules with higher chi-squared value are stored when the identical rules are stored. We have evaluated the proposed method by doing two simulations. Simulation 1 uses fixed threshold values; that is, they remain fixed at initial thresholds during evolution. In simulation 2,thresholds are evolved by mutation in every generation. Fig. 1 shows the number of rules extracted in the pool in simulation 2. It is found that the number of rules extracted has been increased, which means simulation 2 outperforms simulation 1.

The Elevator Group Supervisory Control Systems (EGSCS) are the control systems that systematically manage three or more elevators in order to efficiently transport the passengers in buildings. Double-deck elevators, where two cages are connected with each other, are expected to be the next generation elevator systems. Meanwhile, Destination Floor Guidance Systems (DFGS) are also expected in Double-Deck Elevator Systems (DDES). With these, the passengers could be served at two consecutive floors and could input their destinations at elevator halls instead of conventional systems without DFGS. Such systems become more complex than the traditional systems and require new control methods Genetic Network Programming (GNP), a graph-based evolutionary method, has been applied to EGSCS and its advantages are shown in some previous papers. GNP can obtain the strategy of a new hall call assignment to the optimal elevator because it performs crossover and mutation operations to judgment nodes and processing nodes. In studies so far, the passenger's arrival has been assumed to take Exponential distribution for many years. In this paper, we have applied Erlang distribution and Binomial distribution in order to study how the passenger's arrival distribution affects EGSCS. We have found that the passenger's arrival distribution has great influence on EGSCS. It has been also clarified that GNP makes good performances under different conditions.

This paper proposes a graph-based evolutionary algorithm called Genetic Network Programming (GNP). Our goal is to develop GNP, which can deal with dynamic environments efficiently and effectively, based on the distinguished expression ability of the graph (network) structure. The characteristics of GNP are as follows. 1) GNP programs are composed of a number of nodes which execute simple judgment/processing, and these nodes are connected by directed links to each other. 2) The graph structure enables GNP to re-use nodes, thus the structure can be very compact. 3) The node transition of GNP is executed according to its node connections without any terminal nodes, thus the past history of the node transition affects the current node to be used and this characteristic works as an implicit memory function. These structural characteristics are useful for dealing with dynamic environments. Furthermore, we propose an extended algorithm, "GNP with Reinforcement Learning (GNPRL)" which combines evolution and reinforcement learning in order to create effective graph structures and obtain better results in dynamic environments. In this paper, we applied GNP to the problem of determining agents' behavior to evaluate its effectiveness. Tileworld was used as the simulation environment. The results show some advantages for GNP over conventional methods.

A new graph-based evolutionary algorithm named " Genetic Network Programming, GNP" has been already proposed. GNP represents its solutions as graph structures, which can improve the expression ability and performance. In addition, GNP with Reinforcement Learning (GNP-RL) was proposed a few years ago. Since GNP-RL can do reinforcement learning during task execution in addition to evolution after task execution, it can search for solutions efficiently. In this paper, GNP with Actor-Critic (GNP-AC) which is a new type of GNP-RL is proposed. Originally, GNP deals with discrete information, but GNP-AC aims to deal with continuous information. The proposed method is applied to the controller of the Khepera simulator and its performance is evaluated.

A new graph-based evolutionary algorithm called Genetic Network Programming (GNP) has been proposed. The solutions of GNP are represented as graph structures, which can improve the expression ability and performance. In addition, GNP with Reinforcement Learning (GNP-RL) has been proposed to search for solutions efficiently. GNP-RL can use current information and change its programs during task execution, i. e., online learning. Thus, it has an advantage over evolution-based algorithms in case much information can be obtained during task execution. GNP-RL has a special state-action space and it contributes to reducing the size of the Q-table and learning efficiently. The proposed method is applied to the controller of Khepera simulator and its performance is evaluated.

In this paper, we propose RasID-GA (an abbreviation of Adaptive Random Search with Intensification and Diversification combined with Genetic Algorithm) which improves the ability of diversification searching with Simplex Crossover (SPX). SPX generates the offspring based on uniform probability distribution and uses the M +1 number of parent vectors, where M is the dimension of the vector. The RasID-GA with Simplex Crossover is compared with parallel RasIDs and GA with Simplex Crossover using 23 different objective functions having no local minima, a small number of local minima and a large number of local minima.

Elevator Group Supervisory Control System (EGSCS) is a very large scale stochastic dynamic optimization problem. Due to its vast state space, significant uncertainty, and numerous resource constraints such as finite car capacities and registered hall/car calls, it is hard to manage EGSCS using conventional control methods. Recently, many solutions for EGSCS using Artificial Intelligence (AI) technologies have been reported. Genetic Network Programming (GNP), which is proposed as a new evolutionary computation method several years ago, is also proved to be efficient when applied to EGSCS problem. In this paper, we propose an extended algorithm for EGSCS by introducing Reinforcement Learning (RL) into GNP framework, and expect to make an improvement of the EGSCS’ performances since the efficiency of GNP with RL has been clarified in some other studies like tile-world problem. Simulation tests using traffic flows in a typical office building have been made, and the results show an actual improvement of the EGSCS’ performances comparing to the algorithms using original GNP and conventional control methods.

This paper proposes an improved model for the application of support vector machine (SVM) to achieve the real-world data classification. Being different from traditional SVM classifiers, the new model takes the thought about fuzzy theory into account. And a fuzzy decision-making function is also built to replace the sign function in the prediction stage of classification process. In the prediction part, the method proposed uses the decision value as the independent variable of fuzzy decision-making function to classify test data set into different classes, but not only the sign of which. This flexible design of decision-making model more approaches to the properties of real-world conditions in which interaction and noise influence exist around the boundary between different clusters. So many misclassified cases can be modified when these sets are considered as fuzzy ones. In addition, a boundary offset is also introduced to modify the excursion produced by the imbalance of real-world dataset. Then an improved and more robust performance will be presented by using this adjustable fuzzy decision-making SVM model in simulations.

Inspired by Hebb's cell assembly theory about how the brain worked, we have developed a function localization neural network (FLNN). The main part of a FLNN is structurally the same as an ordinary feedforward neural network, but it is considered to consist of several overlapping modules, which are switched according to input patterns. A FLNN constructed in this way has been shown to have better representation ability than an ordinary neural network. However, BP training algorithm for such FLNN is very easy to get stuck at a local minimum. In this paper, we mainly discuss the methods for improving BP training of the FLNN by utilizing the structural property of the network. Two methods are proposed. Numerical simulations are used to show the effectiveness of the improved BP training methods.

An efficient algorithm for important class association rule mining using genetic network programming (GNP) is proposed. GNP is one of the evolutionary optimization techniques, which uses directed graph structures as genes. Instead of generating a large number of candidate rules, the method can obtain a sufficient number of important association rules for classification. The proposed method measures the significance of the association via the chi-squared test. Therefore, all the extracted important rules can be used for classification directly. In addition, the method suits class association rule mining from dense databases, where many frequently occurring items are found in each tuple. Users can define conditions of extracting important class association rules. In this paper, we describe an algorithm for class association rule mining with chi-squared test using GNP and present a classifier using these extracted rules.

Multiagent Systems with Symbiotic Learning and Evolution (Masbiole) has been proposed and studied, which is a new methodology of Multiagent Systems (MAS) based on symbiosis in the ecosystem. Masbiole employs a method of symbiotic learning and evolution where agents can learn or evolve according to their symbiotic relations toward others, i.e., considering the benefits/losses of both itself and an opponent. As a result, Masbiole can escape from Nash Equilibria and obtain better performances than conventional MAS where agents consider only their own benefits. This paper focuses on the evolutionary model of Masbiole, and its characteristics are examined especially with an emphasis on the behaviors of agents obtained by symbiotic evolution. In the simulations, two ideas suitable for the effective analysis of such behaviors are introduced; "Match Type Tile-world (MTT)" and "Genetic Network Programming (GNP)". MTT is a virtual model where tile-world is improved so that agents can behave considering their symbiotic relations. GNP is a newly developed evolutionary computation which has the directed graph type gene structure and enables to analyze the decision making mechanism of agents easily. Simulation results show that Masbiole can obtain various kinds of behaviors and better performances than conventional MAS in MTT by evolution.

Genetic network programming (GNP) whose gene consists of directed graphs has been proposed as a new method of evolutionary computations, and it is recently applied to the elevator group supervisory control system (EGSCS), a real world problem, to confirm its effectiveness. In the previous study, although the flow of traffic in the elevator system is known and fixed, it is changed dynamically with time in real elevator systems. Therefore, the EGSCS with an adaptive control should be studied considering such changes for practical applications. In this paper, the GNP with functional localization is applied to the EGSCS to construct such an adaptive system. In the proposed method, the switching GNP can switch the functionally localized GNPs (assigning GNPs) fitted to several kinds of traffic by detecting the change of the flow of traffic. From the simulations, the adaptability and effectiveness of the proposed method are clarified using the traffic data of a day in an office building

A novel optimization method named RasID-GA (an abbreviation of adaptive random search with intensification and diversification combined with genetic algorithm) is proposed in order to enhance the searching ability of conventional RasID, which is a kind of random search with intensification and diversification. RasID-GA is compared with conventional RasID and GA using 23 different objective functions, and it turns out that RasID-GA performs well compared with other methods.

Since genetic network programming (GNP) has been proposed as a new method of evolutionary computation, many studies have been done on its applications which cover not only virtual world problems but also real world systems like elevator group supervisory control system (EGSCS) which is a very large scale stochastic dynamic optimization problem. From those researches, most of the significant features of GNP have been verified comparing to genetic algorithm (GA) and genetic programming (GP). Especially, the improvement of the performances on EGSCS using GNP showed an interesting and promising prospect in this field. On the other hand, some studies based on GNP with reinforcement learning (RL) revealed a better performance over conventional GNP on some problems such as tile-world models. As a basic study, reinforcement learning is introduced in this paper expecting to enhance EGSCS controller using GNP

Many methods of generating behavior sequences of agents by evolution have been reported. A new evolutionary computation method named genetic network programming (GNP) has also been developed recently along with these trends. The aim of this paper is to build an artificial model to realize functional localization based on GNP considering the fact that the functional localization of the brain is realized in such a way that a different part of the brain corresponds to a different function. GNP has a directed graph structure suitable for realizing functional localization. In this paper, it is especially stated that the evolution of the switching function can be realized for functional localization of GNP using the self-sufficient garbage collector problem.

According to Hebb's cell assembly theory, the brain has the capability of function localization. On the other hand, it is suggested that the brain has three different learning paradigms: supervised, unsupervised and reinforcement learning. Inspired by the above knowledge of brain, we present a self-organizing function localization neural network (FLNN), that contains supervised, unsupervised and reinforcement learning paradigms. In this paper, we concentrate our discussion mainly on applying a simplified reinforcement learning called evaluative feedback to optimization of the self-organizing FLNN. Numerical simulations show that the self-organizing FLNN has superior performance to an ordinary artificial neural network (ANN).

Recently, artificial neural networks (ANNs) have been utilized for financial market applications. On the other hand, we have so far shown that multi-branch neural networks (MBNNs) could have higher representation and generalization ability than conventional NNs. In this paper, we investigate the accuracy of prediction of TOPIX (Tokyo stock exchange prices indexes) using MBNNs. Using the TOPIX related values in time series and other information, MBNNs can learn the characteristics of time series and predict the TOPIX values of the next day. Several simulations were carried out in order to compare the proposed predictor using MBNNs with that using conventional NNs. The results show that the proposed method can have higher accuracy of the prediction.

Genetic network programming (GNP) has been proposed as a new method of evolutionary computation. Until now, GNP has been applied to various problems and its effectiveness was clarified. However, these problems were virtual models, so the applicability and availability of GNP to the real-world applications have not been studied. In this paper, as a first step of applying GNP to the real-world applications, elevator group supervisory control systems (EGSCSs) are considered. Generally, EGSCSs are complex and difficult problems to solve because they are too dynamic and probabilistic. So the design of a useful controller of EGSCSs was very difficult. Recently, the design of such a controller of EGSCSs has been tried actively using artificial intelligence (AI) technologies. In this paper, it is reported that the design of a controller of EGSCSs has been studied using GNP whose characteristic is to use directed graph as its gene instead of bit strings and trees of GA and GP. From simulations, it is clarified that better solutions are obtained by using GNP than other conventional methods and the availability of GNP to real-world applications is confirmed.

According to the knowledge of brain science, it is suggested that there exists cerebral functional localization, which means that a specific part of the cerebrum is activated depending on various kinds of information human receives. The aim of this paper is to build an artificial model to realize functional localization based on genetic network programming (GNP), a new evolutionary computation method recently developed. GNP has a directed graph structure suitable for realizing functional localization. We studied the basic characteristics of the proposed system by making GNP work in a functionally localized way.

Genetic network programming (GNP) has directed graph structures as genes, which is extended from other evolutionary computations such as genetic algorithm (GA) and genetic programming (GP). Generally, macroinstructions are introduced as sub-routines, function localization and so on. Previously, we have introduced the structure of macroinstructions in GNP named automatically generated macro nodes (AGMs) for reducing the time of evolution efficiently, and showed that macroinstructions are useful to acquire good performances. But the AGMs have fixed number of nodes, and it is found that the effectiveness of evolution of macroinstructions depends on the main program calling them and initialized parameters. Accordingly in this paper, new AGMs are introduced to improve their performances further more by the mechanism of varying the size of AGMs, which are named variable size AGMs. This is the mechanism to add and delete nodes according to necessity. In the simulations, comparisons between GNP program only, GNP with conventional AGMs and GNP with variable size AGMs are carried out using the tile world. Simulation results show that the proposed method is better compared with conventional GNP and GNP with conventional GMs. And also it is clarified that the node transition rules obtained by new AGMs show the generalized rules able to deal with unknown environments.

This work presents a new robust controller design method for nonlinear system based on feedback error learning (FEL) method and higher order derivatives of universal learning networks (ULNs). Our idea is to make an inverse model robust to signal noise by adding the sensitivity terms to the standard criterion function. Through feedback error learning, the sensitivity term can be minimized as well as usual criterion functions using the higher order derivatives of ULNs. As a result, it is confirmed by using simulation results that NNC robust against signal noise can be obtained.

A new architecture of neural networks (NNs) is proposed named neural networks with branch gates (NN-bg). It aims at improving the generalization ability of NNs by controlling the connectivity of neurons adaptively depending on the input information. To realize such architecture, we use a branch control system having low calculation costs. In the branch control system, the distance between the input values of the network and parameters of the branch control system is calculated. After normalization within 0 to 1, the outputs of the branch control system are multiplied to the branches of the NN. The parameters of the branch control system are trained by a random searching method, RasID, to realize an adaptive optimization with very small number of training steps. Through some simulations, the usefulness of the three-layered NN-bg is shown compared with conventional layered neural networks.

Stability is one of the most important subjects in control systems. As for the stability of nonlinear dynamical systems, Lyapunov's direct method and linearized stability analysis method have been widely used. However, finding an appropriate Lyapunov function is fairly difficult, especially for complex nonlinear dynamical systems. Also, it is hard to obtain the locally asymptotically stable region (R/sub LAS/) by these methods. Therefore, it is highly motivated to develop a new stability analysis method that can obtain R/sub LAS/ easily. Accordingly, in this paper, a new stability analysis method based on the higher ordered derivatives (HODs) of universal learning networks (ULNs) with /spl xi/ approximation and its application to a DC motor system are described. The proposed stability analysis method is carried out through two steps: firstly, calculating the first ordered derivatives of any node of the trajectory with respect to the initial disturbances and checking if their values approach zero at time infinity or not. If they approach zero, then the trajectory is locally asymptotically stable. Secondly, obtaining R/sub LAS/, where the first order terms of Taylor expansion are dominant compared to the second order terms with /spl xi/ approximation.

This paper presents a self-organizing function localization neural network (FLNN) inspired by Hebb's cell assembly theory about how the brain worked. The proposed self-organizing FLNN consists of two parts: main part and control part. The main part is an ordinary 3-layered feedforward neural network, but each hidden neuron contains a signal from the control part, controlling its firing strength. The control part consists of a SOM network whose outputs are associated with the hidden neurons of the main part. Trained with an unsupervised learning, SOM control part extracts structural features of input-output spaces and controls the firing strength of hidden neurons in the main part. Such self-organizing FLNN realizes capabilities of function localization and learning. Numerical simulations show that the self-organizing FLNN has superior performance than an ordinary neural network.

Neural networks (NNs) can solve only a simple problem if the network size is too compact, on the other hand, if the network size increases, it costs a lot in terms of calculation time. So, we have studied how to construct the network structure with high performances and low costs in space and time. A solution is a multi-branch structure. Conventional NNs uses the single-branch for the connections, while the multi-branch structure has multi-branches between the nodes. In this paper, a new method which enable the multi-branch NNs to have localized property is proposed. It is well known that RBF networks have localized property that makes it possible to approximate functions faster than sigmoidal NNs. By using the multi-branch structure having localized property, NNs could obtain high performances keeping the lower costs in space and time. Simulation results of function approximations and classification problems illustrated the effectiveness of multi-branch NNs.

Multiagent systems with symbiotic learning and evolution (Masbiole) has been proposed as a new learning and evolutionary method for multiagent systems (MAS) recently, which is based on symbiotic phenomena among creatures. A symbiotic evolutional model of Masbiole is proposed using genetic network programming (GNP), which has been also proposed as one of the evolutionary computations. In the simulations, the proposal Masbiole is applied to the tile-world model and various characteristics of Masbiole have been clarified.

A new evolutionary algorithm named " genetic network programming, GNP" has been proposed. GNP represents its solutions as network structures, which can improve the expression and search ability. Since GA, GP, and GNP already proposed are based on evolution and they cannot change their solutions until one generation ends, we propose GNP with learning and evolution in order to adapt to a dynamical environment quickly. Learning algorithm improves search speed for solutions and evolutionary algorithm enables GNP to search wide solution space efficiently.

Multi-branch neural networks have been proposed already to realize compact networks. In this paper, branch control is proposed on the multi-branch neural networks to further enhance the learning and generalization ability of the networks. Branch control is to adjust the values of the signals on the branches depending on the network inputs using an additional branch control network.

A new evolutionary computation method called genetic network programming (GNP) was proposed recently. In this paper, an online learning method for GNP is proposed. This method uses Q learning to improve its state transition rules so that it can make GNP adapt to dynamic environments efficiently.

Recently, many methods of evolutionary computation such as genetic algorithm (GA) and genetic programming (GP) have been developed as a basic tool for modeling and optimizing of complex systems. Generally speaking, GA has the genome of a string structure, while the genome in GP is the tree structure. Therefore, GP is suitable for constructing complicated programs, which can be applied to many real world problems. However, GP might sometimes be difficult to search for a solution because of its bloat. A novel evolutionary method named Genetic Network Programming (GNP), whose genome is a network structure is proposed to overcome the low searching efficiency of GP and is applied to the problem of the evolution of ant behavior in order to study the effectiveness of GNP. In addition, the comparison of the performances between GNP and GP is carried out in simulations on ant behaviors.

A functions distributed network, called universal learning networks with branch control of relative strength (ULNs with BR), is proposed. The point of the paper is to adjust the outputs of the intermediate nodes of the basic network using an additional branch control network. The adjustment multiplies the nodes outputs by the coefficients ranging from zero to one, which is obtained from the branch control network. Therefore, the following are characterized in ULNs with BR, (1) the branch is cut when the coefficient of its branch is zero, and (2) multiplication is carried out in the nodes outputs adjustment when the coefficient takes a nonzero value. ULNs with BR is applied to two-spirals problem. The simulation results show that ULNs with BR exhibits better performance than the conventional neural networks with comparable complexity.

A new control method of nonlinear dynamic systems is proposed based on the impulse responses of universal learning networks (ULNs), ULNs form a superset of neural networks. They consist of a number of interconnected nodes where the nodes may have any continuously differentiable nonlinear functions in them and each pair of nodes can be connected by multiple branches with arbitrary time delays. A generalized learning algorithm is derived for the ULNs, in which both the first order derivatives (gradients) and the higher order derivatives are incorporated. One of the distinguished features of the proposed control method is that the impulse response of the systems is considered as an extended part of the criterion function and it can be calculated by using the higher order derivatives of ULNs. By using the impulse response as the criterion function, nonlinear dynamics with not only quick response but also quick damping and small steady state error can be more easily obtained than the conventional nonlinear control systems with quadratic form criterion functions of state and control variables.

In this paper, a new genetic symbiosis algorithm (GSA) is proposed based on the symbiotic concept found widely in ecosystems. Since in the conventional genetic algorithms (GA) reproduction is done using only the fitness function of each individual, there are some problems such as premature convergence to an undesirable solution at a very early stage of generation. In addition in some GA applications, it is sometimes required to maintain diversified solutions and to find out many locally optimal solutions. GSA is proposed to solve these problems by considering mutual symbiotic relations between individuals. From simulations on optimizing a nonlinear function, it has been clarified that GSA can find more flexible solutions that can meet a variety of user's requests than the conventional methods.

A min-max robust control method is proposed for nonlinear systems based on the use of the higher order derivatives calculation of universal learning networks (ULNs). An extended criterion function containing sensitivity terms is considered for controller design and the criterion function is evaluated at several specific operating points corresponding to certain system parameters. The ULNs learning is then performed in such a way that, at each step, it minimizes the worst criterion function among several operating points. It is found that the proposed control method is less time-consuming in the ULNs learning and a obtained controller has better performance than the conventional methods.

Universal learning networks with branch control (BrcULNs) are proposed, which consist of basic networks and branch control networks. The branch control network can be used to determine which branches of the basic network should be connected or disconnected. This determination depends on the inputs or the network flows of the basic network. Therefore, by using the BrcULNs, locally functions distributed networks can be realized depending on the values of the inputs of the network or the information of the network flows. The proposed network is applied to some function approximation problems. The simulation results show that the BrcULNs exhibit better performance than the conventional networks with comparable complexity.

Presents a case study for the overlapped multi-neural-network (OMNN). An overlapped multi-neural-network, structurally, is the same as an ordinary feedforward neural network, but it is considered as one consisting of several subnets. All subnets have the same input-output units, but some different hidden units. Input-output spaces are partitioned into several parts, each of which corresponds to one subnet of OMNN. Numerical simulations show that such an OMNN has superior performance in that it has better presentation ability than an ordinary neural network and better generalization ability than a non-overlapped multi-neural-network.

Using the backpropagation algorithm (BP) to train neural networks is a widely adopted practice in both theory and practical applications. However, its distributed weight representation, that is the weight matrix of final network after training by using BP are usually not sparsified, and prohibits its use in the rule discovery of inherent functional relations between the input and output data, so in this aspect some kinds of structure optimization are needed to improve its poor performance. In this paper, with this in mind, a new method to prune neural networks is proposed based on some statistical quantities of neural networks. Comparing with the other known pruning methods such as the structural learning with forgetting and RPROP algorithm, the proposed method can attain comparable or even better results over these methods without evident increase of the computational load. Detailed simulations using the Iris data sets exhibit our above assertion.

Recently many studies have been made on the automatic design of complex systems using evolutionary optimization techniques such as genetic algorithms (GA), evolution strategy (ES), evolutionary programming (EP) and genetic programming (GP). It is generally recognized that these techniques are very useful for optimizing fairly complex systems such as the generation of intelligent behavior sequences of robots. A new method, genetic network programming (GNP), is proposed in order to acquire these behavior sequences efficiently. GNP is composed of plural nodes for agents to execute simple judgment/processing and they are connected with each other to form a network structure. Agents behave according to the contents of the nodes and their connections in GNP. In order to obtain a better structure, the GNP changes itself using evolutionary optimization techniques.

S. A. Kauffman (1993) explored the law of self-organization in random Boolean networks, and K. Inagaki (1998) also did it in neural networks partially. The aim of the paper is to show that probabilistic neural networks (PNNs) hold the order, even though the weights, the thresholds, and the connections between neurons are determined randomly; PNNs are recurrent networks and controlled by a probabilistic transition rule based on a Boltzmann machine. In addition, the deterministic transient neural networks (DNNs) which are the special networks of PNNs are studied extensively. From simulations, it is shown that in DNNs the dynamics follow the square-root law and there is another new critical point for the distribution of the thresholds. In addition, it is shown that in PNNs the averages of the Hamming distance between the attractors of DNN and PNN stay around a certain value depending on the thresholds and the gradient of the Sigmoidal function. These results can be explained by the sensitivity to the initial conditions of DNNs.

This paper applies a quasi-ARMAX modeling technique, presented in the literature, to a process control framework. The use of this quasi-ARMAX modeling technique in nonlinear model predictive control (NMPC) formulations applied to simple nonlinear process control examples is investigated. The quasi-ARMAX predictor can be interpreted as a neuro-fuzzy predictor, and this neuro-fuzzy predictor is computationally straightforward and has shown excellent prediction capabilities. The predictor is thus well suited for NMPC purposes. Furthermore, the parameters of the neuro-fuzzy model can be argued to have explicit meaning, thus making the procedure of tuning the NMPC system more transparent when using the neuro-fuzzy predictor.

Universal learning networks (ULNs) and robust control system design are discussed, ULNs provide a generalized framework to model and control complex systems. They consist of a number of interconnected nodes where the nodes may have any continuously differentiable nonlinear functions in them and each pair of nodes can be connected by multiple branches with arbitrary time delays. Therefore, physical systems which can be described by differential or difference equations and also their controllers can be modeled in a unified way. So, ULNs constitute a superset of neural networks or fuzzy neural networks. In order to optimize the systems, a generalized learning algorithm is derived for the ULNs, in which both the first order derivatives (gradients) and the higher order derivatives are incorporated. The derivatives are calculated by using forward or backward propagation schemes. These algorithms for calculating the derivatives are extended versions of back propagation through time (BPTT) and real time recurrent learning (RTRL) by Williams in the sense that generalized nonlinear functions and higher order derivatives are dealt with. As an application of ULNs, the higher order derivative, one of the distinguished features of ULNs, is applied to realizing a robust control system in this paper. In addition, it is shown that the higher order derivatives are effective tools to realize sophisticated control of nonlinear systems. Other features of ULNs such as multiple branches with arbitrary time delays and using a priori information will be discussed in other papers.

A new chaos control method is proposed which is useful for taking advantage of chaos and avoiding it. The proposed method is based on the following facts: (1) chaotic phenomena can be generated and eliminated by controlling the maximum Lyapunov exponent of the systems, and (2) the maximum Lyapunov exponent can be formulated and calculated by using higher-order derivatives of universal learning networks (ULNs). ULNs consist of a number of interconnected nodes which may have any continuously differentiable nonlinear functions in them and where each pair of nodes can be connected by multiple branches with arbitrary time delays. A generalized learning algorithm has been derived for the ULNs in which both first-order derivatives (gradients) and higher-order derivatives are incorporated. In simulations, parameters of ULNs with bounded node outputs were adjusted for the maximum Lyapunov exponent to approach the target value, and it has been shown that a fully-connected ULN with three sigmoidal function nodes is able to generate and eliminate chaotic behaviors by adjusting these parameters.

The universal learning network (ULN) which is a superset of supervised learning networks has been already proposed. Parameters in ULN are trained in order to optimize a criterion function as conventional neural networks, and after training they are used as constant parameters. In this paper, a method to alter the parameters depending on the network flows is presented to enhance representation abilities of networks. In the proposed method, there exists two kinds of networks, the first one is a basic network which includes varying parameters and the other one is a network which calculates the optimal varying parameters depending on the network flows of the basic network. It is also proposed in this paper that any type of networks such as fuzzy inference networks, radial basis function networks and neural networks can be used for the basic and parameter calculation networks. From simulations where parameters in a neural network are altered by fuzzy inference networks, it is shown that the networks with the same number of varying parameters have higher representation abilities than the conventional networks.

A method for identifying nonlinear dynamic systems with noise is proposed by using probabilistic universal learning networks (PrULNs). PrULNs are extensions of universal learning networks (ULNs). ULNs form a superset of neural networks and were proposed to provide a universal framework for modeling and control of nonlinear large-scale complex systems. But the ULN does not provide any stochastic characteristics of the signals propagating through it. The PrULNs are equipped with machinery to calculate stochastic properties of signals and to train network parameters so that the signals behave with the pre-specified stochastic properties. On the other hand it is generally recognized that there exists an overfitting problem when identification of nonlinear dynamic systems with noise is done by neural networks. In this paper, it is shown from simulation results of identification of a nonlinear robot dynamics that PrULNs are useful for avoiding the overfitting.

Introduces a scheme to construct object oriented learning networks (OOLN). The idea is to construct a learning network with two levels. The high-level network is built based on application specific prior knowledge such that it has a structure favorable to the applications. The low-level one consists of a class of conventional neural networks. The OOLN is expected to have both application flexibility and representation flexibility.

In this paper, it is studied how the generalization ability of modeling of the dynamic systems can be improved by taking advantages of the second order derivatives of the criterion function with respect to the external inputs. The proposed method is based on the regularization theory proposed by Poggio and Givosi (1990), but a main distinctive point in this paper is that extension to dynamic systems from static systems has been taken into account and actual second order derivatives of the universal learning network have been used to train the parameters of the networks. The second order derivatives term of the criterion function may minimize the deviation caused by the external input changes. Simulation results show that the method is useful for improving the generalization ability of identifying nonlinear dynamic systems using neural networks.

Universal learning networks (ULNs) have been proposed, which are a super set of any kinds of supervised learning networks. One of the important features of the ULNs is that the ULNs have a systematic algorithm for calculating higher order derivatives of the criterion function with respect to parameters. Both robust control and chaos control methods using the second order derivatives of the ULNs have also been proposed. In this paper, a new control design method of the nonlinear systems is proposed, which is an extension of the above robust control method in terms of the stability as well as the quick response of the systems.

Chaotic behaviors are characterized mainly by Lyapunov numbers of a dynamic system. In this paper, a new method is proposed, which can control the maximum Lyapunov number of dynamic system that can be represented by universal learning networks (ULNs). The maximum Lyapunov number of a dynamic system can be formulated by using higher order derivatives of ULNs and parameters of ULNs can be adjusted for the maximum Lyapunov number to approach the target value by the combined gradient and random search method. Based on simulation results, a fully connected ULN with three nodes is possible to display chaotic behaviors.

Recently, a number of studies have been done to investigate complicated systems such as economical, social, ecological and living systems. And, it is well known that complicated systems can be analyzed by the concept of symbiosis which is made up of competition, exploitation and coexistence and so on. The purpose of this paper is to propose a new modeling method for complicated systems by using the concept of symbiosis. A useful modeling method is presented in order to improve the ability of representing the symbiosis phenomena using the technique of fuzzy inference.

According to recent knowledge of brain science it is suggested that there exists functions distribution, which means that specific parts exist in the brain for realizing specific functions. This paper introduces a new brain-like model called Learning Petri Network (LPN) that has the capability of functions distribution and learning. The idea is to use Petri net to realize the functions distribution and to incorporate the learning and representing ability of neural network into the Petri net. The obtained LPN can be used in the same way as a neural network to model and control dynamic systems, while it is distinctive to a neural network in that it has the capability of functions distribution. An application of the LPN to nonlinear crane control systems is discussed. It is shown via numerical simulations that the proposed LPN controller has superior performance to the commonly-used neural network one.