Power efficiency is plateauing in the standard digital electronics realm such that new hardware, models, and algorithms are needed to reduce the costs of AI training. The combination of energy-based analog circuits and the Equilibrium Propagation (EP) algorithm constitutes a compelling alternative compute paradigm for gradient-based optimization of neural nets. Existing analog hardware accelerators, however, typically incorporate digital circuitry to sustain auxiliary non-weight-stationary operations, mitigate analog device imperfections, and leverage existing digital platforms. Such heterogeneous hardware lacks a supporting theoretical framework. In this work, we introduce \emph{Feedforward-tied Energy-based Models} (ff-EBMs), a hybrid model comprised of feedforward and energy-based blocks housed on digital and analog circuits. We derive a novel algorithm to compute gradients end-to-end in ff-EBMs by backpropagating and ``eq-propagating'' through feedforward and energy-based parts respectively, enabling EP to be applied flexibly on realistic architectures. We experimentally demonstrate the effectiveness of this approach on ff-EBMs using Deep Hopfield Networks (DHNs) as energy-based blocks, and show that a standard DHN can be arbitrarily split into any uniform size while maintaining or improving performance with increases in simulation speed of up to four times. We then train ff-EBMs on ImageNet32 where we establish a new state-of-the-art performance for the EP literature (46 top-1 \%). Our approach offers a principled, scalable, and incremental roadmap for the gradual integration of self-trainable analog computational primitives into existing digital accelerators.

Despite being the focus of a thriving field of research, the biological mechanisms that underlie information integration in the brain are not yet fully understood. A theory that has gained a lot of traction in recent years suggests that multi-scale integration is regulated by a hierarchy of mutually interacting neural oscillations. In particular, there is accumulating evidence that phase-amplitude coupling (PAC), a specific form of cross-frequency interaction, plays a key role in numerous cognitive processes. Current research in the field is not only hampered by the absence of a gold standard for PAC analysis, but also by the computational costs of running exhaustive computations on large and high-dimensional electrophysiological brain signals. In addition, various signal properties and analyses parameters can lead to spurious PAC. Here, we present Tensorpac, an open-source Python toolbox dedicated to PAC analysis of neurophysiological data. The advantages of Tensorpac include (1) higher computational efficiency thanks to software design that combines tensor computations and parallel computing, (2) the implementation of all most widely used PAC methods in one package, (3) the statistical analysis of PAC measures, and (4) extended PAC visualization capabilities. Tensorpac is distributed under a BSD-3-Clause license and can be launched on any operating system (Linux, OSX and Windows). It can be installed directly via pip or downloaded from Github (https://github.com/EtienneCmb/tensorpac). By making Tensorpac available, we aim to enhance the reproducibility and quality of PAC research, and provide open tools that will accelerate future method development in neuroscience.