Local Optima Networks (LON)s have been used extensively to understand the global structure of optimisation problems and to study algorithm behaviour. The central idea is to compress the search space into a graph object capturing the local optima along with information on their basins of attraction and the connections between them. This enables the visualisation of high dimensional search spaces, and the extraction of metrics for characterising and contrasting different problem instances. In this paper we extend the canonical LON definition to encompass search spaces with constraints. We use a well-known pairwise comparison operator for constrained problems, and capture the features of the constraint violation landscape that present a challenge for such an operator, such as infeasible local traps. The concept of a constrained LON is illustrated through a range of problem instances. Most problems in the context of real-world applications have constraints. By including the notion of feasibility and constraint violation into the definition of LONs, it becomes possible to use this powerful analysis tool on a much wider range of real-world problems.

The global structure of the hyperparameter spaces of neural networks is not well understood and it is therefore not clear which hyperparameter search algorithm will be most effective. In this paper we analyze the landscapes of convolutional neural network architecture search spaces to provide insight into appropriate search algorithms for these spaces. Using a classical fitness landscape analysis approach (fitness distance correlation) and a more recent tool (local optima networks) we study the global structure of these spaces. Our analysis on six image classification datasets reveals that the landscapes are multi-modal, but with relatively few local optima from which it is not hard to escape with a simple perturbation operator. This led us to explore the performance of iterated local search, which we found to more effectively search the training landscapes than three evolutionary algorithm variants. Evolutionary algorithms, however, outperformed iterated local search in terms of generalization on problems with larger discrepancies between the training and testing landscapes.

The notion and characterisation of fitness landscapes has helped us understand the performance of heuristic algorithms on complex optimisation problems. Many practical problems, however, are constrained, and when significant areas of the search space are infeasible, researchers have intuitively resorted to a variety of constraint-handling techniques intended to help the algorithm manoeuvre through infeasible areas and toward feasible regions of better fitness. It is clear that providing constraint-related feedback to the algorithm to influence its choice of solutions overlays the violation landscape with the fitness landscape in unpredictable ways whose effects on the algorithm cannot be directly measured. In this work, we apply metrics of violation landscapes to continuous and combinatorial problems to characterise them. We relate this information to the relative performance of six well-known constraint-handling techniques to demonstrate how some properties of constrained landscapes favour particular constraint-handling approaches. For the problems with sampled feasible solutions, a bi-objective approach was the best performing approach overall, but other techniques performed better on problems with the most disjoint feasible areas. For the problems with no measurable feasibility, a feasibility ranking approach was the best performing approach overall, but other techniques performed better when the correlation between fitness values and the level of constraint violation was high.

The characterisation of topographical features of fitness landscapes can provide significant insight into the nature of underlying optimisation problems and the behaviour of metaheuristic search algorithms. Neutrality as a landscape feature is often overlooked in continuous problems, but researchers have theorised that the presence of neutral regions on neural network error surfaces may be an impediment to current population-based search algorithms for training neural networks. An empirical approach to measuring the amount of neutrality would provide a stepping stone for future studies on the effects of neutrality. To date, there is no offline technique to achieve this in continuous domains. This paper proposes two normalised measures of neutrality based on a progressive random walk algorithm. Measurements are shown to agree with visual analysis of two-dimensional benchmark problems, and are shown to scale well to higher dimensions. The measures are ultimately applied to neural network classification problems where saturation-induced neutrality is confirmed.

Artificial neural networks are inherently high-dimensional, which limits our ability to visualise and understand their inner workings. Neural network architecture and training algorithm parameters are usually optimised on an ad hoc basis, with very limited insight into the nature of the objective function landscape. This study proposes using fitness landscape analysis to quantify topological properties of neural network error landscapes. Five techniques from the fitness landscape analysis field are adapted to work with neural network error landscapes. These techniques are then used to analyse how the error landscape changes under different error measurements and different number of hidden layers. The results show that fitness landscape analysis provides valuable insight into neural network error landscapes, and could be used for architecture selection.

Real-world optimisation problems are usually constrained in some way. These constraints essentially modify the search space and can have a significant impact on the success of algorithms during optimisation. This paper proposes the notion of a violation landscape as a concept for analysing the nature of constrained continuous search spaces. A number of numerical measures are proposed for characterising constrained problems and these are tested on the CEC 2010 benchmark suite of constrained real-parameter optimisation problems. It is shown that for many constrained problems and algorithms, the features of the violation landscape are more relevant in terms of understanding algorithm performance than the features of the fitness landscape.

A number of fitness landscape analysis approaches are based on random walks through discrete search spaces. Applying these approaches to real-encoded problems requires the notion of a random walk in continuous space. This paper proposes a progressive random walk algorithm and the use of multiple walks to sample neighbourhood structure in continuous multi-dimensional spaces. It is shown that better coverage of a search space is provided by progressive random walks than simple unbiased random walks.

Portfolio optimization has as its objective to find optimal portfolios, which apportion capital between their constituent assets such that the portfolio's risk adjusted return is maximized. Portfolio optimization becomes more complex as constraints are imposed, multiple sources of return are included, and alternative measures of risk are used. Meta-heuristic portfolio optimization can be used as an alternative to deterministic approaches under increased complexity conditions. This paper uses a particle swarm optimization (PSO) algorithm to optimize a diversified portfolio of carry trades. In a carry trade, investors profit by borrowing low interest rate currencies and lending high interest rate currencies, thereby generating return through the interest rate differential. However, carry trades are risky because of their exposure to foreign exchange losses. Previous studies showed that diversification does significantly mitigate this risk. This paper goes one step further and shows that meta-heuristic portfolio optimization can further improve the risk adjusted returns of diversified carry trade portfolios.

Particle swarm optimisation (PSO) algorithms have been successfully used to solve many complex real-world optimisation problems. Since their introduction in 1995, the focus of research in PSOs has largely been on the algorithmic side with many new variations proposed on the original PSO algorithm. Relatively little attention has been paid to the study of problems with respect to PSO performance. The aim of this study is to investigate whether a link can be found between problem characteristics and algorithm performance for PSOs. A range of benchmark problems are numerically characterised using fitness landscape analysis techniques. Decision tree induction is used to develop failure prediction models for seven different variations on the PSO algorithm. Results show that for most PSO models, failure could be predicted to a fairly high level of accuracy. The resulting prediction models are not only useful as predictors of failure, but also provide insight into the algorithms themselves, especially when expressed as fuzzy rules in terms of fitness landscape features.

Fitness landscape analysis has focussed on many different aspects of optimisation problems such as ruggedness, neutrality, epistasis and evolvability. Although many techniques have been proposed, there are very few that have been shown to be practically useful as predictors of algorithm performance. This paper investigates three metrics related to the structure of fitness landscapes of continuous problems: a ruggedness measure based on entropy, a dispersion index measure for detecting the presence of funnels and a new proposed technique for estimating gradients. Results on a range of benchmark problems show that all proposed measures show some correlation to performance of a traditional particle swarm optimisation (PSO) algorithm on the same benchmark problems. The three metrics could therefore have value as part-predictors of PSO performance on unknown problems if used in conjunction with measures approximating other features that have been linked to problem difficulty for PSOs.

The vector evaluated particle swarm optimization (VEPSO) algorithm is a cooperative, multi-swarm algorithm. Each sub-swarm optimizes only a single objective of a multi-objective problem (MOP), and implements a knowledge transfer strategy (KTS) to share optimal positions of the different objectives among the sub-swarms, guiding the particles to different regions of the Pareto front. This paper shows that the stagnation problem that occurs in VEPSO can be addressed by using a different KTS. A comparison is made between the ring-based and random knowledge transfer strategies. Experimental results show that the random knowledge transfer strategy suffers less from stagnation than the ring-based KTS, making it the preferred KTS to use.

This paper investigates different trade decision strategies under different market conditions so that a genetic algorithm could be designed to use the appropriate decision strategy. A trade decision strategy defines how a single action is decided upon based on a number of signals where each signal is a result of a technical analysis function. Using historical market data, a population is trained using a simple genetic algorithm employing crossover and mutation. Four genetic algorithms are used to evolve agents to trade, where each genetic algorithm uses a different trade decision strategy. The best individual from each evolved population is compared using an out-of-sample data set. Results show a significant difference in performance between the four decision strategies especially within bearish to moderately bullish stocks. Populations evolved using a weighted decision strategy performs better than strategies that are not weighted when trading bearish to moderately bullish stocks. Non-weighted decision strategies appear to out-perform weighted strategies when used on extremely bullish stock. This out-performance could be attributed to fewer trades made by non-weighted strategies compared to weighted ones.

A major unsolved problem in the field of optimisation and computational intelligence is how to determine which algorithms are best suited to solving which problems. This research aims to analytically characterise individual problems as a first step towards attempting to link problem types with the algorithms best suited to solving them. In particular, an information theoretic technique for analysing the ruggedness of a fitness landscape with respect to neutrality was adapted to work in continuous landscapes and to output a single measure of ruggedness. Experiments run on test functions with increasing ruggedness show that the proposed measure of ruggedness produced relative values consistent with a visual inspection of the problem landscapes. Combined with other measures of complexity, the proposed ruggedness measure could be used to more broadly characterise the complexity of fitness landscapes in continuous domains.

This paper investigates the feasibility of using Genetic Programming in dynamically changing environments to evolve decision trees for classification problems and proposes an new version of Genetic Programming called Adaptive Genetic Programming. It does so by comparing the performance or classification error of Genetic Programming and Adaptive Genetic Programming to that of Gradient Descent in abruptly and progressively changing environments. To cope with dynamic environments, Adaptive Genetic Programming incorporates adaptive control parameters, variable elitism and culling. Results show that both Genetic Programming and Adaptive Genetic Programming are viable algorithms for dynamic environments yielding a performance gain over Gradient Descent for lower dimensional problems even with severe environment changes. In addition, Adaptive Genetic Programming performs slightly better than Genetic Programming, due to faster recovery from changes in the environment.

In studies that compare the performance of population-based optimization algorithms, it is sometimes assumed that the comparison is valid as long as the number of function evaluations is equal, even if the population size differs. This paper shows that such comparisons are invalid. The performance of two algorithms: differential evolution (DE) and global best particle swarm optimization (gbest PSO) are tested on standard benchmark problems with different numbers of individuals/particles (20, 50 and 100). It is shown that there are significance differences in the performance of the same algorithm with the same number of function evaluations, but with different numbers of individuals/particles. Comparisons of different algorithms should therefore always use the same population size for results to be valid.

Some multimedia archives contain data which have vague locations in time and space. By this we mean that, although there is some idea of when and where the entity is located, the precise information is unknown. In this paper, we present a novel approach to displaying and querying such uncertain data. We use the concepts of dynamic queries, add to this a 2D query tool for performing spatial queries and enable Boolean combinations of queries. We have implemented these ideas in a pilot system for querying African artwork. In this way, we show how it is possible for novice users to easily query large multimedia archives with complex uncertain attributes.