The choice of a proper learning rate is paramount for good Artificial Neural Network training and performance. In the past, one had to rely on experience and trial-and-error to find an adequate learning rate. Presently, a plethora of state of the art automatic methods exist that make the search for a good learning rate easier. While these techniques are effective and have yielded good results over the years, they are general solutions. This means the optimization of learning rate for specific network topologies remains largely unexplored. This work presents AutoLR, a framework that evolves Learning Rate Schedulers for a specific Neural Network Architecture using Structured Grammatical Evolution. The system was used to evolve learning rate policies that were compared with a commonly used baseline value for learning rate. Results show that training performed using certain evolved policies is more efficient than the established baseline and suggest that this approach is a viable means of improving a neural network's performance.

Developing learning systems is challenging in many ways: often there is the need to optimise the learning algorithm structure and parameters, and it is necessary to decide which is the best data representation to use, i.e., we usually have to design features and select the most representative and useful ones. In this work we focus on the later and investigate whether or not it is possible to obtain good performances with compressed versions of the original data, possibly reducing the learning time. The process of compressing the data, i.e., reducing its dimensionality, is typically conducted by someone who has domain knowledge and expertise, and engineers features in a trial-and-error endless cycle. Our goal is to achieve such compressed versions automatically; for that, we use an Evolutionary Algorithm to generate the structure of AutoEncoders. Instead of targeting the reconstruction of the images, we focus on the reconstruction of the mean signal of each class, and therefore the goal is to acquire the most representative characteristics of each class. Results on the MNIST dataset show that the proposed approach can not only reduce the original dataset dimensionality, but the performance of the classifiers over the compressed representation is superior to the performance on the original uncompressed images.

The effectiveness of Artificial Neural Networks (ANNs) depends on a non-trivial manual crafting of their topology and parameters. Typically, practitioners resort to a time consuming methodology of trial-and-error to find and/or adjust the models to solve specific tasks. To minimise this burden one might resort to algorithms for the automatic selection of the most appropriate properties of a given ANN. A remarkable example of such methodologies is Grammar-based Genetic Programming. This work analyses and compares the use of two grammar-based methods, Grammatical Evolution (GE) and Structured Grammatical Evolution (SGE), to automatically design and configure ANNs. The evolved networks are used to tackle several classification datasets. Experimental results show that SGE is able to automatically build better models than GE, and that are competitive with the state of the art, outperforming hand-designed ANNs in all the used benchmarks.