If the trend of learned components eventually outperforming their hand-crafted version continues, learned optimizers will eventually outperform hand-crafted optimizers like SGD or Adam. Even if learned optimizers (L2Os) eventually outpace hand-crafted ones in practice however, they are still not provably convergent and might fail out of distribution. These are the questions addressed here. Currently, learned optimizers frequently outperform generic hand-crafted optimizers (such as gradient descent) at the beginning of learning but they generally plateau after some time while the generic algorithms continue to make progress and often overtake the learned algorithm as Aesop’s tortoise which overtakes the hare. L2Os also still have a difficult time generalizing out of distribution. \cite{heaton_safeguarded_2020} proposed Safeguarded L2O (GL2O) which can take a learned optimizer and safeguard it with a generic learning algorithm so that by conditionally switching between the two, the resulting algorithm is provably convergent. We propose a new class of Safeguarded L2O, called Loss-Guarded L2O (LGL2O), which is both conceptually simpler and computationally less expensive. The guarding mechanism decides solely based on the expected future loss value of both optimizers. Furthermore, we show theoretical proof of LGL2O’s convergence guarantee and empirical results comparing to GL2O and other baselines showing that it combines the best of both L2O and SGD and that in practice converges much better than GL2O.

Presented topic is from the research fields called Artificial Life and Artificial Intelligence (AI). In this paper, there is presented novel approach to designing agent architectures with its requirements. The approach is inspired by inherited modularity of biological brains and agent architectures are represented here as set of given reusable modules connected into a particular topology. This paper presents design of two particular modules for future use in more complex architectures. The modules are used for implementing model-free motivation-driven Reinforcement Learning (RL). First, the novel framework for these architectures is described together with a used simulator. Then, the design of two new reusable domain-independent components of agent architectures is described. Finally, experimental validation of these new components and their future use is mentioned.

We propose our original system capable of autonomous design of general-purpose complex modular hybrid systems. The resulting hybrid systems will be able to employ various techniques of learning, decisionmaking, prediction etc. Presented topic is from Artificial Life domain, but contributes also to fields such as Artificial Intelligence, Biology, Computational Neuroscience, Ethology, Cybernetics and potentially into many other aspects of research. The autonomous design is implemented as an optimization of system topology with respect to given problem. The principle of design is based on modified neuro-evolution and can be compared to modular neural networks. One of the main requirements is standardization of communication between very different subsystems. Here, each subsystem module implements arbitrary algorithm and is treated as a Multiple-Input Multiple-Output subsystem. First, the design of simulator used is described, then the basic principle of hybrid networks is explained with it benefits and drawbacks. Finally, simple example is mentioned.

Presented topic is from area of development of artificial creatures and proposes new architecture of autonomous agent. The work builds on a research of the latest approaches to Artificial Life, realized by the Department of Cybernetics, CTU in Prague in the last twenty years. This architecture design combines knowledge from Artificial Intelligence (AI), Ethology, Artificial Life (ALife) and Intelligent Robotics. From the field of classical AI, the fusion of reinforcement learning, planning and artificial neural network into one more complex control system was used here. The main principle of its function is inspired by the field of Ethology, this means that life of given agent tries to be similar to life of an animal in the Nature, where animal learns relatively autonomously from simpler principles towards the more complex ones. The architecture supports on-line learning of all knowledge from the scratch, while the core principle is in hierarchical Reinforcement Learning (RL), this action hierarchy is created autonomously based solely on agents interaction with an environment. The main key idea behind this approach is in original implementation of a domain independent hierarchical planner. Our planner is able to operate with behaviors learned by the RL. It means that an autonomously gained hierarchy of actions can be used not only by action selection mechanisms based on the reinforcement learning, but also by a planning system. This gives the agent ability to utilize high-level deliberative problem solving based solely on his experiences. In order to deal with higher-level control rather than a sensory system, the life of agent was simulated in a virtual environment.