MANBIS is a C++ mathematical software package for tackling the problem of computing the roots of a function when the number of roots is very large (of the order of hundreds or thousands). This problem has attracted increasing attention in recent years because of the broad variety of applications in various fields of science and technology. MANBIS applies the bisection method to obtain an approximate root according to a predetermined accuracy. Thus, the only computable information required is the algebraic signs of the considered function, which is the smallest amount of information (one bit of information) necessary for the purpose needed, and not any additional information. MANBIS is able to compute very efficiently a user-given percentage of roots and draws its strength from the fact that the roots are expected to be many. Furthermore, MANBIS is capable of estimating without any additional function computational cost the total number of roots within the user-given interval. Our approach can also be efficiently applied in cases where the distribution of the roots is not known. This article is accompanied by another article where the user manual, some implementation details, and some examples are included.

Training a multilayer perceptron (MLP) with algorithms employing global search strategies has been an important research direction in the field of neural networks. Despite a number of significant results, an important matter concerning the bounds of the search region---typically defined as a box---where a global optimization method has to search for a potential global minimizer seems to be unresolved. The approach presented in this paper builds on interval analysis and attempts to define guaranteed bounds in the search space prior to applying a global search algorithm for training an MLP. These bounds depend on the machine precision and the term guaranteed denotes that the region defined surely encloses weight sets that are global minimizers of the neural network's error function. Although the solution set to the bounding problem for an MLP is in general non-convex, the paper presents the theoretical results that help deriving a box which is a convex set. This box is an outer approximation of the algebraic solutions to the interval equations resulting from the function implemented by the network nodes. An experimental study using well known benchmarks is presented in accordance with the theoretical results.

Reliable estimation of a neural network's domain of validity is important for a number of reasons such as assessing its ability to cope with a given problem, evaluating the consistency of its generalization etc. In this paper we introduce a new approach to estimate the domain of validity of a neural network based on Set Inversion Via Interval Analysis (SIVIA), the methodology established by Jaulin and Walter [1]. This approach was originally introduced in order to solve nonlinear parameter estimation problems in a bounded error context and proved to be effective in tackling several types of problems dealing with nonlinear systems analysis. The dependence of a neural network output on the pattern data is a nonlinear function and hence derivation of the impact of the input data to the neural network function can be addressed as a nonlinear parameter estimation problem that can be tackled by SIVIA. We present concrete application examples and show how the proposed method allows to delimit the domain of validity of a trained neural network. We discuss advantages, pitfalls and potential improvements offered to neural networks.

Breast cancer can be prevented with regular mammography screening. Yet, the incorporation of Computational Intelligence relies on training classifiers on a set of predefined Regions of Interest (ROIs). Data Clustering has been applied to address the problem of ROI detection, yet no extensive research has been carried out on which algorithm to utilize. This contribution focuses on microcalcification clustering as a Data Clustering application, giving insights concerning the performance of three main clustering algorithms.

A new family of Differential Evolution mutation strategies (DE/nrand) that are able to handle multimodal functions, have been recently proposed. The DE/nrand family incorporates information regarding the real nearest neighborhood of each potential solution, which aids them to accurately locate and maintain many global optimizers simultaneously, without the need of additional parameters. However, these strategies have increased computational cost. To alleviate this problem, instead of computing the real nearest neighbor, we incorporate an index-based neighborhood into the mutation strategies. The new mutation strategies are evaluated on eight well-known and widely used multimodal problems and their performance is compared against five state-of-the-art algorithms. Simulation results suggest that the proposed strategies are promising and exhibit competitive behavior, since with a substantial lower computational cost they are able to locate and maintain many global optima throughout the evolution process.

An active research direction in Particle Swarm Optimization (PSO) is the integration of PSO variants in adaptive, or self-adaptive schemes, in an attempt to aggregate their characteristics and their search dynamics. In this work we borrow ideas from adaptive filter theory to develop an “online” algorithm adaptation framework. The proposed framework is based on tracking the parameters of a multinomial distribution to capture changes in the evolutionary process. As such, we design a multinomial distribution tracker to capture the successful evolution movements of three PSO variants. Extensive experimental results on ten benchmark functions and comparisons with five state-of-the-art algorithms indicate that the proposed framework is competitive and very promising. On the majority of tested cases, the proposed framework achieves substantial performance gain, while it seems to identify accurately the most appropriate algorithm for the problem at hand.

Computational Intelligence encompasses tools that allow the fast convergence and adaptation to several problems, a fact that makes them eligible for real-time implementations. The paper at hand discusses the utilization of intelligent algorithms (i.e. Differential Evolution and Genetic Algorithms) for the creation of an adaptive system that is able to provide real-time automatic music accompaniment to a human improviser. The main goal of the presented system is to generate accompanying music based on the local human musician's tonal, rhythmic and intensity playing style, incorporating no prior knowledge about the improvisers intentions. Compared to existing systems previously proposed, this work introduces a constraint-free improvisation environment where the most important musical characteristics are automatically adapted to the human performer's playing style, without any prior information. This fact allows the improviser to have maximal control over the tonal, rhythmic and intensity improvisation directions.

Differential evolution is a very popular optimization algorithm and considerable research has been devoted to the development of efficient search operators. Motivated by the different manner in which various search operators behave, we propose a novel framework based on the proximity characteristics among the individual solutions as they evolve. Our framework incorporates information of neighboring individuals, in an attempt to efficiently guide the evolution of the population toward the global optimum, without sacrificing the search capabilities of the algorithm. More specifically, the random selection of parents during mutation is modified, by assigning to each individual a probability of selection that is inversely proportional to its distance from the mutated individual. The proposed framework can be applied to any mutation strategy with minimal changes. In this paper, we incorporate this framework in the original differential evolution algorithm, as well as other recently proposed differential evolution variants. Through an extensive experimental study, we show that the proposed framework results in enhanced performance for the majority of the benchmark problems studied.

In recent years, the Particle Swarm Optimization has rapidly gained increasing popularity and many variants and hybrid approaches have been proposed to improve it. Motivated by the behavior and the proximity characteristics of the social and cognitive experience of each particle in the swarm, we develop a hybrid approach that combines the Particle Swarm Optimization and the Differential Evolution algorithm. Particle Swarm Optimization has the tendency to distribute the best personal positions of the swarm near to the vicinity of problem's optima. In an attempt to efficiently guide the evolution and enhance the convergence, we evolve the personal experience of the swarm with the Differential Evolution algorithm. Extensive experimental results on twelve high dimensional multimodal benchmark functions indicate that the hybrid variants are very promising and improve the original algorithm.

This papers proposes a novel self-adaptive scheme for the evolution of crucial control parameters in evolutionary algorithms. More specifically, we suggest to utilize the differential evolution algorithm to endemically evolve its own control parameters. To achieve this, two simultaneous instances of Differential Evolution are used, one of which is responsible for the evolution of the crucial user-defined mutation and recombination constants. This self-adaptive differential evolution algorithm alleviates the need of tuning these user-defined parameters while maintains the convergence properties of the original algorithm. The evolutionary self-adaptive scheme is evaluated through several well-known optimization benchmark functions and the experimental results indicate that the proposed approach is promising.

The hybridization and composition of different Evolutionary Algorithms to improve the quality of the solutions and to accelerate execution is a common research practice. In this paper we propose a hybrid approach that combines differential evolution mutation operators in an attempt to balance their exploration and exploitation capabilities. Additionally, a self-balancing hybrid mutation operator is presented, which favors the exploration of the search space during the first phase of the optimization, while later opts for the exploitation to aid convergence to the optimum. Extensive experimental results indicate that the proposed approaches effectively enhance DEpsilas ability to accurately locate solutions in the search space.

A technique for the visualization of stochastic population-based algorithms in multidimensional problems with known global minimizers is proposed. The technique employs projections of the populations in the 2-dimensional vector space spanned by the two extremal eigenvectors of the Hessian matrix of the objective function at a global minimizer. This space condenses information regarding the shape of the objective function around the given minimizer. The proposed approach can provide intuition regarding the behavior of the algorithm in unknown high-dimensional problems. It also provides an alternative visualization framework for problems of any dimension, which alleviates drawbacks of the most popular projection methods. The proposed technique is illustrated for three well-known population-based algorithms, namely, differential evolution, covariance matrix adaptation evolution strategies and particle swarm optimization, on three test problems of different dimensionality.

The Differential Evolution algorithm uses an elitist selection, constantly pushing the population in a strict downhill search, in an attempt to guarantee the conservation of the best individuals. However, when this operator is combined with an exploitive mutation operator can lead to premature convergence to an undesired region of attraction. To alleviate this problem, we propose the Non-Monotone Differential Evolution algorithm. To this end, we allow the best individual to perform some uphill movements, greatly enhancing the exploration of the search space. This approach further aids algorithm's ability to escape undesired regions of the search space and improves its performance. The proposed approach utilizes already computed pieces of information and does not require extra function evaluations. Experimental results indicate that the proposed approach provides stable and reliable convergence.

We propose a Particle Filter model that incorporates Particle Swarm Optimization for predicting systems with multiplicative noise. The proposed model employs a conventional multiobjective optimization approach to weight the likelihood and prior of the filter in order to alleviate the particle impoverishment problem. The resulting scheme is tested on a well-known test problem with multiplicative noise. Results are promising, especially in cases of high system and measurement noise levels.

This paper investigates the performance of trading strategies identified through computational intelligence techniques. We focus on trading rules derived by genetic programming, as well as, generalized moving average rules optimized through differential evolution. The performance of these rules is investigated using recently proposed risk-adjusted evaluation measures and statistical testing is carried out through simulation. Overall, the moving average rules proved to be more robust, but genetic programming seems more promising in terms of generating higher profits and detecting novel patterns in the data.

The computation of periodic orbits of nonlinear mappings is very important for studying and better understanding the dynamics of complex systems. Evolutionary algorithms have shown to be an efficient alternative for the computation of periodic orbits in cases where the inherent properties of the problem at hand render gradient-based methods invalid. Such cases usually involve nondifferentiable mappings or poorly behaved partial derivatives. We propose a Memetic Particle Swarm Optimization algorithm that exploits Shannon’s information entropy for decision making in swarm level, as well as a probabilistic decision making scheme in particle level, for determining when and where local search is applied. These decisions have a significant impact on the required number of function evaluations, especially in cases where high accuracy is desirable. Experimental results are performed on well-known problems and useful conclusions are derived.

Fuzzy cognitive maps constitute an important simulation methodology that combines neural networks and fuzzy logic. The Fuzzy cognitive maps designed by the experts can be enhanced significantly through learning algorithms, which proved to increase their efficiency and accuracy of simulation. Recently, learning algorithms that employ particle swarm optimization for the minimization of properly defined objective functions have been introduced. In this work, we enhance these learning schemes by incorporating local search in PSO, resulting in a memetic particle swarm optimization learning algorithm. Three variants of the memetic algorithm are applied successfully for the optimization of an Ecological Industrial Park simulation system and they are compared also with the established particle swarm optimization learning schemes. Results are reported and discussed, deriving useful conclusions

The hybridization and combination of different Evolutionary Algorithms to improve the quality of the solutions and to accelerate execution is a common research practice. In this paper, we utilize Genetic Programming to evolve novel Differential Evolution operators. The genetic evolution resulted in parameter free Differential Evolution operators. Our experimental results indicate that the performance of the genetically programmed operators is comparable and in some cases is considerably better than the already existing human designed ones.

In this paper a new clustering operator for evolutionary algorithms is proposed. The operator incorporates the unsupervised k-windows clustering algorithm, utilizing already computed pieces of information regarding the search space in an attempt to discover regions containing groups of individuals located close to different minimizers. Consequently, the search is confined inside these regions and a large number of global and local minima of the objective function can be efficiently computed. Extensive experiments shown that the proposed approach is effective and reliable, and greatly accelerates the convergence speed of the considered algorithms.

Evolutionary clustering is a recent trend in cluster analysis that has the potential to yield high partitioning accuracy results. Traditional evolutionary techniques applied in clustering are typically hindered by the high cost involved in the computation of the objective function. In this paper, the authors proposed a novel objective function that can provide fitness function values in sub-linear time. Next an evolutionary scheme was developed to evolve cluster solutions and demonstrate how the number of clusters can be estimated from the final result. Finally, by employing real world datasets, the high quality clustering results that this scheme can provide was shown.

Networks of spiking neurons can perform complex non-linear computations in fast temporal coding just as well as rate coded networks. These networks differ from previous models in that spiking neurons communicate information by the timing, rather than the rate, of spikes. To apply spiking neural networks on particular tasks, a learning process is required. Most existing training algorithms are based on unsupervised Hebbian learning. In this paper, we investigate the performance of the parallel differential evolution algorithm, as a supervised training algorithm for spiking neural networks. The approach was successfully tested on well-known and widely used classification problems.

Recent advances in microarray technologies have allowed scientists to discover and monitor the mRNA transcript levels of thousands of genes in a single experiment. The data obtained from microarray studies present a challenge to data analysis. In this paper, we design an expression-based classification method for acute leukemia. Different dimension reduction techniques are considered to tackle the very high dimensionality of this kind of data. Subsequently, the classification system employs artificial neural networks. The comparative results reported, indicate that high classification rates are possible and moreover that subsets of features that contribute significantly to the success of the neural classifiers can be identified.

We investigate the performance of the recently proposed unified particle swarm optimization algorithm on two categories of operations research problems, namely minimax and integer programming problems. Different variants of the algorithm are employed and compared with established variants of the particle swarm optimization algorithm. Statistical hypothesis testing is performed to justify the significance of the results. Conclusions regarding the ability of the unified particle swarm optimization method to tackle operations research problems as well as on the performance of each variant are derived and discussed.

A parallel, multi-population differential evolution algorithm for multiobjective optimization is introduced. The algorithm is equipped with a domination selection operator to enhance its performance by favouring non-dominated individuals in the populations. Preliminary experimental results on widely used test problems are promising. Comparisons with the VEGA approach are provided and discussed.

Parallel processing has emerged as a key enabling technology in modern computing. Recent software advances have allowed collections of heterogeneous computers to be used as a concurrent computational resource. In this work we explore how differential evolution can be parallelized, using a ring-network topology, so as to improve both the speed and the performance of the method. Experimental results indicate that the extent of information exchange among subpopulations assigned to different processor nodes, bears a significant impact on the performance of the algorithm. Furthermore, not all the mutation strategies of the differential evolution algorithm are equally sensitive to the value of this parameter.

An extension of the well-known probabilistic neural network (PNN), to generalized locally recurrent PNN (GLRPNN) is introduced. This extension renders GLRPNN, in contrast to PNN, sensitive to the context, in which events occur. A GLRPNN is therefore, able to identify time or spatial correlations. This capability can be exploited to improve performance on classification tasks. A fast three-step algorithm for training GLRPNN is also proposed. The first two steps are identical to the training of traditional PNN, while the third step exploits the differential evolution optimization method. The performance of the proposed methodology on the task of text-independent speaker verification is contrasted with that of locally recurrent PNN, diagonal recurrent neural networks, infinite impulse response and finite impulse response MLP-based structures, as well as with a Gaussian mixture models-based classifier.

The most widely used algorithm for training multilayer feedforward neural networks is backpropagation. Back-propagation is an iterative gradient descent algorithm. Since its appearance, various methods which modify the conventional BP have been created to improve its efficiency. One such algorithm which uses an adaptive learning rate is backpropagation with variable stepsize, is proposed. Parallel tangent methods are used in global optimization to modify and improve the simple gradient descent algorithm by using from time to time the difference between the current point and the point before two steps as a search direction, instead of using the gradient. In this study, we investigate the combination of the BPVS method with the parallel tangent approach for neural network training. We perform experimental results on well-known test problems to evaluate the efficiency of the method.

Abstract This article shows that the procedure proposed in Chang and Wu (1997) and extended in Dini (2001) does not always produce accurate results. A modification that makes the procedure correct is suggested. © 2004 Wiley Periodicals, Inc.

This paper presents approaches for effectively computing all global minimizers of an objective function. The approaches include transformations of the objective function through the recently proposed deflection and stretching techniques, as well as a repulsion source at each detected minimizer. The aforementioned techniques are incorporated in the context of the particle swarm optimization (PSO) method, resulting in an efficient algorithm which has the ability to avoid previously detected solutions and, thus, detect all global minimizers of a function. Experimental results on benchmark problems originating from the fields of global optimization, dynamical systems, and game theory, are reported, and conclusions are derived.

During the last decade, numerous heuristic search methods for solving multi-objective optimization problems have been developed. Population oriented approaches such as evolutionary algorithms and particle swarm optimization can be distinguished into the class of archive-based algorithms and algorithms without archive. While the latter may lose the best solutions found so far, archive based algorithms keep track of these solutions. In this article, a new particle swarm optimization technique, called DOPS, for multi-objective optimization problems is proposed. DOPS integrates well-known archiving techniques from evolutionary algorithms into particle swarm optimization. Modifications and extensions of the archiving techniques are empirically analyzed and several test functions are used to illustrate the usability of the proposed approach. A statistical analysis of the obtained results is presented. The article concludes with a discussion of the obtained results as well as ideas for further research.

We introduce a new algorithm for fuzzy cognitive maps learning. The proposed approach is based on the particle swarm optimization method and it is used for the detection of proper weight matrices that lead the fuzzy cognitive map to desired steady states. For this purpose a properly defined objective function that incorporates experts' knowledge is constructed and minimized. The application of the proposed methodology to an industrial control problem supports the claim that the proposed technique is efficient and robust.

The existence of roots of functions is a topic of major significance in nonlinear analysis, and it is directly related to the problem of detection of extrema of a function. The topological degree of a function is a mathematical tool of great importance for investigating the existence and the number of roots of a function with certainty. For the computation of the topological degree according to Stenger's theorem, a sufficient refinement of the boundary of the polyhedron under consideration is needed. The sufficient refinement can be computed using the optimal complexity algorithm of Boult and Sikorski. However, the application of this algorithm requires the computation of the infinity norm on the boundary of the polyhedron under consideration as well as an estimation of the Lipschitz constant of the function. We introduced a new technique for the computation of the infinity norm on the polyhedron's boundary as well as for the estimation of the Lipschitz constant. The proposed approach is illustrated on several test problems and the results are reported and discussed.

We present a time series forecasting methodology and applies it to generate one-step-ahead predictions for two daily foreign exchange spot rate time series. The methodology draws from the disciplines of chaotic time series analysis, clustering, artificial neural networks and evolutionary computation. In brief, clustering is applied to identify neighborhoods in the reconstructed state space of the system; and subsequently neural networks are trained to model the dynamics of each neighborhood separately. The results obtained through this approach are promising.

This paper introduces Locally Recurrent Probabilistic Neural Networks (LRPNN) as an extension of the well-known Probabilistic Neural Networks (PNN). A LRPNN, in contrast to a PNN, is sensitive to the context in which events occur, and therefore, identification of time or spatial correlations is attainable. Besides the definition of the LRPNN architecture a fast three-step training method is proposed. The first two steps are identical to the training of traditional PNNs, while the third step is based on the Differential Evolution optimization method. Finally, the superiority of LRPNNs over PNNs on the task of text-independent speaker verification is demonstrated.

Periodic orbits of nonlinear mappings play a central role in the study of dynamical systems. Traditional root finding algorithms, such as the Newton-family algorithms, have been widely applied for the detection of periodic orbits. However, in the case of discontinuous/nondifferentiable mappings and mappings with poorly behaved partial derivatives, this approach is not valid. In such cases, stochastic optimization algorithms have proved to be a valuable tool. In this paper, a new approach for computing periodic orbits through particle swarm optimization is introduced. The results indicate that the algorithm is robust and efficient. Moreover, the method can be combined with established techniques, such as deflection, to detect several periodic orbits of a mapping. Finally, the minor effort which is required to implement the proposed approach renders it an efficient alternative for computing periodic orbits of nonlinear mappings.

This paper investigates the ability of the Particle Swarm Optimization (PSO) method to cope with minimax problems through experiments on well-known test functions. Experimental results indicate that PSO tackles minimax problems effectively. Moreover, PSO alleviates difficulties that might be encountered by gradient-based methods, due to the nature of the minimax: objective function, and potentially lead to failure. The performance of PSO is compared with that of other established approaches, such as the sequential quadratic programming (SQP) method and a recently proposed smoothing technique.

The investigation of the performance of the particle swarm optimization (PSO) method in integer programming problems, is the main theme of the present paper. Three variants of PSO are compared with the widely used branch and bound technique, on several integer programming test problems. Results indicate that PSO handles efficiently such problems, and in most cases it outperforms the branch and bound technique.

Clustering, that is the partitioning of a set of patterns into disjoint and homogeneous meaningful groups (clusters), is a fundamental process in the practice of science. k-windows is an efficient clustering algorithm that reduces the number of patterns that need to be examined for similarity. using a windowing technique. It exploits well known spatial data structures, namely the range free, that allows fast range searches. From a theoretical standpoint, the k-windows algorithm is characterized by lower time complexity compared to other well-known clustering algorithms. Moreover it achieves high quality clustering results. However, it appears that it cannot be directly applicable in high-dimensional settings due to the superlinear space requirements for the range tree. In this paper an improvement of the k-windows algorithm, aiming at resolving this deficiency, is presented. The improvement is based on an alternative solution to the orthogonal range search problem.

A novel generalized theoretical result is presented that underpins the development of globally convergent first-order batch training algorithms which employ local learning rates. This result allows us to equip algorithms of this class with a strategy for adapting the overall direction of search to a descent one. In this way, a decrease of the batch-error measure at each training iteration is ensured, and convergence of the sequence of weight iterates to a local minimizer of the batch error function is obtained from remote initial weights. The effectiveness of the theoretical result is illustrated in three application examples by comparing two well-known training algorithms with local learning rates to their globally convergent modifications.

We present deterministic nonmonotone learning strategies for multilayer perceptrons (MLPs), i.e., deterministic training algorithms in which error function values are allowed to increase at some epochs. To this end, we argue that the current error function value must satisfy a nonmonotone criterion with respect to the maximum error function value of the M previous epochs, and we propose a subprocedure to dynamically compute M. The nonmonotone strategy can be incorporated in any batch training algorithm and provides fast, stable, and reliable learning. Experimental results in different classes of problems show that this approach improves the convergence speed and success percentage of first-order training algorithms and alleviates the need for fine-tuning problem-depended heuristic parameters.

A new theorem for the development and convergence analysis of supervised training algorithms with an adaptive learning rate for each weight is presented. Based on this theoretical result, a strategy is proposed to automatically adapt the search direction, as well as the step-size length along the resultant search direction. This strategy is applied to some well known local learning algorithms to investigate its effectiveness.

Abstmct- Evolutionary neural network training algorithms are presented. These algorithms are applied to train neural networks with weight values confined to a narrow band of integers. We constrain the weights and biases in the range [-2"-l + 1, 2k-1- 11, for k = 3,4,5, thus they can be represented by just k bits. Such neural networks are better suited for hardware implementation than the real weight ones. Mathematical operations that are easy to implement in software might often be very burdensome in the hardware and therefore more costly. Hardware-friendly algorithms are essential to ensure the functionality and cost effectiveness of the hardware implementation. To this end, in addition to the integer weights, the trained neural networks use threshold activation functions only, so hardware implementation is even easier. These algorithms have been designed keeping in mind that the resulting integer weights require less bits to be stored and the digital arithmetic operations between them are easier to be implemented in hardware. Obviously, if the network is trained in a constrained weight space, smaller weights are found and less memory is required. On the other hand, as we have found here, the network training procedure can be more effective and efficient when larger weights are allowed. Thus, for a given application a trade off between effectiveness and memory consumption has to be considered. Our intention is to present results of evolutionary algorithms on this difficult task. Based on the application of the proposed class of methods on classical neural network benchmarks, our experience is that these methods are effective and reliable. 1

Presents neural network training algorithms which are based on the differential evolution (DE) strategies introduced by Storn and Price (J. of Global Optimization, vol. 11, pp. 341-59, 1997). These strategies are applied to train neural networks with small integer weights. Such neural networks are better suited for hardware implementation than the real weight ones. Furthermore, we constrain the weights and biases in the range [-2/sup k/+1, 2/sup k/-1], for k=3,4,5. Thus, they can be represented by just k bits. These algorithms have been designed keeping in mind that the resulting integer weights require less bits to be stored and the digital arithmetic operations between them are more easily implemented in hardware. Obviously, if the network is trained in a constrained weight space, smaller weights are found and less memory is required. On the other hand, the network training procedure can be more effective and efficient when large weights are allowed. Thus, for a given application, a trade-off between effectiveness and memory consumption has to be considered. We present the results of evolution algorithms for this difficult task. Based on the application of the proposed class of methods on classical neural network benchmarks, our experience is that these methods are effective and reliable.

Training algorithms suitable to work under imprecise conditions are proposed. They require only the algebraic sign of the error function or its gradient to be correct, and depending on the way they update the weights, they are analyzed as composite nonlinear successive overrelaxation (SOR) methods or composite nonlinear Jacobi methods, applied to the gradient of the error function. The local convergence behavior of the proposed algorithms is also studied. The proposed approach seems practically useful when training is affected by technology imperfections, limited precision in operations and data, hardware component variations and environmental changes that cause unpredictable deviations of parameter values from the designed configuration. Therefore, it may be difficult or impossible to obtain very precise values for the error function and the gradient of the error during training.

We present nonmonotone methods for feedforward neural network training, i.e., training methods in which error function values are allowed to increase at some iterations. More specifically, at each epoch we impose that the current error function value must satisfy an Armijo-type criterion, with respect to the maximum error function value of M previous epochs. A strategy to dynamically adapt M is suggested and two training algorithms with adaptive learning rates that successfully employ the above mentioned acceptability criterion are proposed. Experimental results show that the nonmonotone learning strategy improves the convergence speed and the success rate of the methods considered.

A mathematical framework for the convergence analysis of the well known Quickprop method is described. The convergence of this method is analyzed. Furthermore, we present modifications of the algorithm that exhibit improved convergence speed and stability and at the same time, alleviate the use of heuristic learning parameters. Simulations are conducted to compare and evaluate the performance of a proposed modified Quickprop algorithm with various popular training algorithms. The results of the experiments indicate that the increased convergence rates, achieved by the proposed algorithm, affect by no means its generalization capability and stability.

This article focuses on gradient-based backpropagation algorithms that use either a common adaptive learning rate for all weights or an individual adaptive learning rate for each weight and apply the Goldstein/Armijo line search. The learning-rate adaptation is based on descent techniques and estimates of the local Lipschitz constant that are obtained without additional error function and gradient evaluations. The proposed algorithms improve the backpropagation training in terms of both convergence rate and convergence characteristics, such as stable learning and robustness to oscillations. Simulations are conducted to compare and evaluate the convergence behavior of these gradient-based training algorithms with several popular training methods.

Two algorithms are described here for the numerical solution of a system of nonlinear equations F(X) = Θ, Θ(0,0,…,0)∈ ℝ, and F is a given continuous mapping of a region 𝒟 in ℝ n into ℝ n . The first algorithm locates at least one root of the sy stem within n -dimensional polyhedron, using the non zero v alue of the topological degree of F at θ relative to the polyhedron; th e second algorithm applies a new generalized bisection method in order to compute an approximate solution to the system. Teh size of the original n -dimensional polyhedron is arbitrary, and the method is globally convergent in a residual sense. These algorithms, in the various function evaluations, only make use of the algebraic sign of F and do not require computations of the topological degree. Moreover, they can be applied to nondifferentiable continuous functions F and do not involve derivatives of F or approximations of such derivatives.

CHABIS is a mathematical software package for the numerical solution of a system of n nonlinear equations in n variables. First, CHABIS locates at least one solution of the system within an n -dimensional polyhedron. Then, it applies a new generalized method of bisection to this n -polyhedron in order to obtain an approximate solution of the system according to a predetermined accuracy. In this paper we briefly describe the user interface to CHABIS and present several details of its implementation, as well as an example of its usage.