Performance prediction has been a key part of the neural architecture search (NAS) process, allowing to speed up NAS algorithms by avoiding resource-consuming network training. Although many performance predictors correlate well with ground truth performance, they require training data in the form of trained networks. Recently, zero-cost proxies have been proposed as an efficient method to estimate network performance without any training. However, they are still poorly understood, exhibit biases with network properties, and their performance is limited. Inspired by the drawbacks of zero-cost proxies, we propose neural graph features (GRAF), simple to compute properties of architectural graphs. GRAF offers fast and interpretable performance prediction while outperforming zero-cost proxies and other common encodings. In combination with other zero-cost proxies, GRAF outperforms most existing performance predictors at a fraction of the cost.

The energy efficiency of hardware implementations of convolutional neural networks (CNNs) is critical to their widespread deployment in low-power mobile devices. Recently, a number of methods have been proposed for providing energy-optimal mappings of CNNs onto diverse hardware accelerators. Their estimated energy consumption is related to specific implementation details and hardware parameters, which does not allow for machine-independent exploration of CNN energy measures. In this letter, we introduce a simplified theoretical energy complexity model for CNNs, based on only a two-level memory hierarchy that captures asymptotically all important sources of energy consumption for different CNN hardware implementations. In this model, we derive a simple energy lower bound and calculate the energy complexity of evaluating a CNN layer for two common data flows, providing corresponding upper bounds. According to statistical tests, the theoretical energy upper and lower bounds we present fit asymptotically very well with the real energy consumption of CNN implementations on the Simba and Eyeriss hardware platforms, estimated by the Timeloop/Accelergy program, which validates the proposed energy complexity model for CNNs.

Centroid-based methods have an established place in a variety of tasks including object localization in images. A sophisticated method for constructing optimal centroids and corresponding weights has been proposed only recently. In order to reduce the computational demands of applying the optimal centroid, several novel sparse versions of the optimal centroids are proposed here, which are based on trimming away some of their pixels. Suitable novel sparse versions bring improvements compared to available optimal centroids. At the same time, some of the sparse optimal centroids (especially the method with thresholded optimal weights) turn out to be robust to noise in the images.

Metalearning, an important part of artificial intelligence, represents a promising approach for the task of automatic selection of appropriate methods or algorithms. This paper is interested in recommending a suitable estimator for nonlinear regression modeling, particularly in recommending either the standard nonlinear least squares estimator or one of such available alternative estimators, which is highly robust with respect to the presence of outliers in the data. The authors hold the opinion that theoretical considerations will never be able to formulate such recommendations for the nonlinear regression context. Instead, metalearning is explored here as an original approach suitable for this task. In this paper, four different approaches for automatic method selection for nonlinear regression are proposed and computations over a training database of 643 real publicly available datasets are performed. Particularly, while the metalearning results may be harmed by the imbalanced number of groups, an effective approach yields much improved results, performing a novel combination of supervised feature selection by random forest and oversampling by synthetic minority oversampling technique (SMOTE). As a by-product, the computations bring arguments in favor of the very recent nonlinear least weighted squares estimator, which turns out to outperform other (and much more renowned) estimators in a quite large percentage of datasets.

Deep neural networks enjoy high interest and have become the state-of-art methods in many fields of machine learning recently.Still, there is no easy way for a choice of network architecture.However, the choice of architecture can significantly influence the network performance.This work is the first step towards an automatic architecture design.We propose a genetic algorithm for an optimization of a network architecture.The algorithm is inspired by and designed directly for the Keras library [1] that is one of the most common implementations of deep neural networks.The target application is the prediction of air pollution based on sensor measurements.The proposed algorithm is evaluated on experiments on sensor data and compared to several fixed architectures and support vector regression.

Kernel-based neural networks are popular machine learning approach with many successful applications. Regularization networks represent a their special subclass with solid theoretical background and a variety of learning possibilities. In this paper, we focus on single and multi-kernel units, in particular, we describe the architecture of a product unit network, and describe an evolutionary learning algorithm for setting its parameters including different kernels from a dictionary, and optimal split of inputs into individual products. The approach is tested on real-world data from calibration of air-pollution sensor networks, and the performance is compared to several different regression tools.

This paper deals with learning possibilities of regularization networks with product kernel units. Approximation problems formulated as regularized minimization problems with kernel-based stabilizers lead to solutions of the shape of linear combination of kernel functions. These can be expressed as one-hidden layer feed-forward neural network schemes, called regularization networks. We propose a novel evolutionary algorithm utilizing for regularization networks with product kernels. This algorithm utilizes genetic search for suitable network parameters as well as kernel functions.

An emergence of intelligent behaviour within a simple robotic agent is studied in this paper. The radial basis function neural network is used as the control mechanism of the robot. Evolutionary algorithm is used to train the agent to perform several tasks. A comparison to multilayer perceptron neural networks and reinforcement learning is made and the results are discussed.