Adapting to regularities of the environment is critical for biological organisms to anticipate events and plan. A prominent example is the circadian rhythm corresponding to the internalization by organisms of the $24$-hour period of the Earth’s rotation. In this work, we study the emergence of circadian-like rhythms in deep reinforcement learning agents. In particular, we deployed agents in an environment with a reliable periodic variation while solving a foraging task. We systematically characterize the agent’s behavior during learning and demonstrate the emergence of a rhythm that is endogenous and entrainable. Interestingly, the internal rhythm adapts to shifts in the phase of the environmental signal without any re-training. Furthermore, we show via bifurcation and phase response curve analyses how artificial neurons develop dynamics to support the internalization of the environmental rhythm. From a dynamical systems view, we demonstrate that the adaptation proceeds by the emergence of a stable periodic orbit in the neuron dynamics with a phase response that allows an optimal phase synchronisation between the agent’s dynamics and the environmental rhythm.

Recent experimental findings indicate that Purkinje cells in the cerebellum represent time intervals by mechanisms other than conventional synaptic weights. These findings add to the theoretical and experimental observations suggesting the presence of intra-cellular mechanisms for adaptation and processing. To account for these experimental results we propose a new biophysical model for time interval learning in a Purkinje cell. The numerical model focuses on a classical delay conditioning task (e.g. eyeblink conditioning) and relies on a few computational steps. In particular, the model posits the activation by the parallel fiber input of a local intra-cellular calcium store which can be modulated by intra-cellular pathways. The reciprocal interaction of the calcium signal with several proteins forming negative and positive feedback loops ensures that the timing of inhibition in the Purkinje cell anticipates the interval between parallel and climbing fiber inputs during training. We systematically test the model ability to learn time intervals at the 150-1000 ms time scale, while observing that learning can also extend to the multiple seconds scale. In agreement with experimental observations we also show that the number of pairings required to learn increases with inter-stimulus interval. Finally, we discuss how this model would allow the cerebellum to detect and generate specific spatio-temporal patterns, a classical theory for cerebellar function.