The remarkable success of the transformer machine learning architecture for processing language sequences far exceeds the performance of classical signal processing methods. A unique component of transformer models is the scaled dot-product attention (SDPA) layer, which does not appear to have an analog in prior signal processing algorithms. Here, we show that SDPA operates using a novel principle that projects the current state estimate onto the space spanned by prior estimates. We show that SDPA, when used for causal recursive state estimation, implements constrained state estimation in circumstances where the constraint is unknown and may be time varying. Since constraints in high-dimensional space may represent the complex relationships that define nonlinear signals and models, this suggests that the SDPA layer and transformer models leverage constrained estimation to achieve their success. This also suggests that transformers and the SPDA layer could be a computational model for previously unexplained capabilities of human behavior.

Children and adults with cerebral palsy (CP) can have involuntary upper limb movements as a consequence of the symptoms that characterize their motor disability, leading to difficulties in communicating with caretakers and peers. We describe how a socially assistive robot may help individuals with CP to practice non-verbal communicative gestures using an active orthosis in a one-on-one number-guessing game. We performed a user study and data collection with participants with CP; we found that participants preferred an embodied robot over a screen-based agent, and we used the participant data to train personalized models of participant engagement dynamics that can be used to select personalized robot actions. Our work highlights the benefit of personalized models in the engagement of users with CP with a socially assistive robot and offers design insights for future work in this area.

The cerebellum is known to have an important role in sensing and execution of precise time intervals, but the mechanism by which arbitrary time intervals can be recognized and replicated with high precision is unknown. We propose a computational model in which precise time intervals can be identified from the pattern of individual spike activity in a population of parallel fibers in the cerebellar cortex. The model depends on the presence of repeatable sequences of spikes in response to conditioned stimulus input. We emulate granule cells using a population of Izhikevich neuron approximations driven by random but repeatable mossy fiber input. We emulate long-term depression (LTD) and long-term potentiation (LTP) synaptic plasticity at the parallel fiber to Purkinje cell synapse. We simulate a delay conditioning paradigm with a conditioned stimulus (CS) presented to the mossy fibers and an unconditioned stimulus (US) some time later issued to the Purkinje cells as a teaching signal. We show that Purkinje cells rapidly adapt to decrease firing probability following onset of the CS only at the interval for which the US had occurred. We suggest that detection of replicable spike patterns provides an accurate and easily learned timing structure that could be an important mechanism for behaviors that require identification and production of precise time intervals.

Human movement differs from robot control because of its flexibility in unknown environments, robustness to perturbation, and tolerance of unknown parameters and unpredictable variability. We propose a new theory, risk-aware control, in which movement is governed by estimates of risk based on uncertainty about the current state and knowledge of the cost of errors. We demonstrate the existence of a feedback control law that implements risk-aware control and show that this control law can be directly implemented by populations of spiking neurons. Simulated examples of risk-aware control for time-varying cost functions as well as learning of unknown dynamics in a stochastic risky environment are provided.

Our central goal is to quantify the long-term progression of pediatric neurological diseases, such as a typical 10-15 years progression of child dystonia. To this purpose, quantitative models are convincing only if they can provide multi-scale details ranging from neuron spikes to limb biomechanics. The models also need to be evaluated in hyper-time, i.e. significantly faster than real-time, for producing useful predictions. We designed a platform with digital VLSI hardware for multi-scale hyper-time emulations of human motor nervous systems. The platform is constructed on a scalable, distributed array of Field Programmable Gate Array (FPGA) devices. All devices operate asynchronously with 1 millisecond time granularity, and the overall system is accelerated to 365x real-time. Each physiological component is implemented using models from well documented studies and can be flexibly modified. Thus the validity of emulation can be easily advised by neurophysiologists and clinicians. For maximizing the speed of emulation, all calculations are implemented in combinational logic instead of clocked iterative circuits. This paper presents the methodology of building FPGA modules in correspondence to components of a monosynaptic spinal loop. Results of emulated activities are shown. The paper also discusses the rationale of approximating neural circuitry by organizing neurons with sparse interconnections. In conclusion, our platform allows introducing various abnormalities into the neural emulation such that the emerging motor symptoms can be analyzed. It compels us to test the origins of childhood motor disorders and predict their long-term progressions.

Control in the natural environment is difficult in part because of uncertainty in the effect of actions. Uncertainty can be due to added motor or sensory noise, unmodeled dynamics, or quantization of sensory feedback. Biological systems are faced with further difficulties, since control must be performed by networks of cooperating neurons and neural subsystems. Here, we propose a new mathematical framework for modeling and simulation of distributed control systems operating in an uncertain environment. Stochastic differential operators can be derived from the stochastic differential equation describing a system, and they map the current state density into the differential of the state density. Unlike discrete-time Markov update operators, stochastic differential operators combine linearly for a large class of linear and nonlinear systems, and therefore the combined effects of multiple controllable and uncontrollable subsystems can be predicted. Design using these operators yields systems whose statistical behavior can be specified throughout state-space. The relationship to Bayesian estimation and discrete-time Markov processes is described.

In this paper we analyze stochastic stability and boundedness of the neurophysiologically inspired feedback error learning (FEL) paradigm, a control algorithm that uses an inverse model of the plant to maximize tracking performance under uncertain conditions. FEL is analyzed in the framework of an adaptive state feedback controller. An inverse model of the plant is adaptively learned by a neural network based on basis functions, while the output of the feedback controller is used as the training signal. The nonlinear plant under consideration is described as a multidimensional SISO stochastic differential equation. The tracking error was shown to be uniformly bounded in the case where the variance of the noise on the parameter update rule was constant and the variance of the noise on the state variables was a function of the tracking error. When the system was allowed to have only noise on the states variables, with variance linear to the tracking error, then FEL was shown to be stochastically stable

This paper examines the feedback error learning architecture that has been proposed by Kawato under specific conditions in which learning does not occur. When a robot attempts to learn a novel task in an unknown environment, an approximation of the inverse dynamics of the plant/environment may be required. The novelty of feedback error learning lies in the training signal used to iteratively construct the inverse feedforward controller to achieve this task. We discuss a class of failure modes where the interaction of the learning algorithm and the feedback control leads to poor performance despite repeated practice. We hypothesize that this model could describe motor learning failure commonly seen in childhood movement disorders where a task, such as reaching in a straight path to an intended target, is never learned or improved despite years of repeated practice.

For certain complex motor tasks, humans may experience the frustration of a lack of improvement despite repeated practice. We investigate a computational basis for failure of motor learning when there is no prior information about the system to be controlled and when it is not practical to perform a thorough random exploration of the set of possible commands. In this case, if the desired movement has never yet been performed, then it may not be possible to learn the correct motor commands since there will be no appropriate training examples. We derive the mathematical basis for this phenomenon when the controller can be modeled as a linear combination of nonlinear basis functions trained using a gradient descent learning rule on the observed commands and their results. We show that there are two failure modes for which continued training examples will never lead to improvement in performance. We suggest that this may provide a model for the lack of improvement in human skills that can occur despite repeated practice of a complex task.

This article proposes a new method for interpreting computations performed by populations of spiking neurons. Neural firing is modeled as a rate-modulated random process for which the behavior of a neuron in response to external input can be completely described by its tuning function. I show that under certain conditions, cells with any desired tuning functions can be approximated using only spike coincidence detectors and linear operations on the spike output of existing cells. I show examples of adaptive algorithms based on only spike data that cause the underlying cell-tuning curves to converge according to standard supervised and unsupervised learning algorithms. Unsupervised learning based on principal components analysis leads to independent cell spike trains. These results suggest a duality relationship between the random discrete behavior of spiking cells and the deterministic smooth behavior of their tuning functions. Classical neural network approximation methods and learning algorithms based on continuous variables can thus be implemented within networks of spiking neurons without the need to make numerical estimates of the intermediate cell firing rates.

The theory of Optimal Unsupervised Motor Learning shows how a network can discover a reduced-order controller for an unknown nonlinear system by representing only the most significant modes. Here, I extend the theory to apply to command sequences, so that the most significant components discovered by the network corre(cid:173) spond to motion "primitives". Combinations of these primitives can be used to produce a wide variety of different movements. I demonstrate applications to human handwriting decomposition and synthesis, as well as to the analysis of electrophysiological experiments on movements resulting from stimulation of the frog spinal cord.

Recent evidence of population coding in motor cortex has led some researchers to claim that certain variables such as hand direction or force may be coded within a Cartesian coordinate system with respect to extra personal space. These claims are based on the ability to predict the rectangular coordinates of hand movement direction using a “population vector” computed from multiple cells' firing rates. I show here that such a population vector can always be found given a very general set of assumptions. Therefore the existence of a population vector constitutes only weak support for the explicit use of a particular coordinate representation by motor cortex.

In this paper, optimal unsupervised motor learning is defined to be a technique for finding the coordinate system of minimum dimensionality which can adequately describe a particular motor task. An explicit method is provided for learning a stable controller that translates commands within the new coordinate system into motor variables appropriate for plant control. The method makes use of previously described neural network algorithms including the generalized Hebbian algorithm, basis-function trees, and trajectory extension learning. Examples of applications to a real direct-drive two joint planar robot arm and a simulated three joint robot arm with visual sensing are given.

Trajectory extension learning is an incremental method for training an artificial neural network to approximate the inverse dynamics of a robot manipulator. Training data near a desired trajectory is obtained by slowly varying a parameter of the trajectory from a region of easy solvability of the inverse dynamics toward the desired behavior. The parameter can be average speed, path shape, feedback gain, or any other controllable variable. As learning proceeds, an approximate solution to the local inverse dynamics for each value of the parameter is used to guide learning for the next value of the parameter. Convergence conditions are given for two variations on the algorithm. Examples are shown of application to a real 2-joint direct drive robot arm and a simulated 3-joint redundant arm, both using simulated equilibrium point control.>

The output layer of a feedforward neural network approximates nonlinear functions as a linear combination of a fixed set of basis functions, or "features". These features are learned by the hidden-layer units, often by a supervised algorithm such as a back-propagation algorithm. This paper investigates features which are optimal for computing desired output functions from a given distribution of input data, and which must therefore be learned using a mixed supervised and unsupervised algorithm. A definition is proposed for optimal nonlinear features, and a constructive method, which has an iterative implementation, is derived for finding them. The learning algorithm always converges to a global optimum and the resulting network uses two layers to compute the hidden units. The general form of the features is derived for the case of continuous signal input, and this result is related to transmission of information through a bandlimited channel. The results of other algorithms can he compared to the optimal features, which in some cases have easily computed closed-form solutions. The application of this technique to the inverse kinematics problem for a simulated planar two-joint robot arm is demonstrated here.

I describe a new algorithm for approximating continuous functions in high-dimensional input spaces. The algorithm builds a tree-structured network of variable size, which is determined both by the distribution of the input data and by the function to be approximated. Unlike other tree-structured algorithms, learning occurs through completely local mechanisms and the weights and structure are modified incrementally as data arrives. Efficient computation in the tree structure takes advantage of the potential for low-order dependencies between the output and the individual dimensions of the input. This algorithm is related to the ideas behind k-d trees (Bentley 1975), CART (Breiman et al. 1984), and MARS (Friedman 1988). I present an example that predicts future values of the Mackey-Glass differential delay equation.

Nonlinear function approximation is often solved by finding a set of coefficients for a finite number of fixed nonlinear basis functions. However, if the input data are drawn from a high-dimensional space, the number of required basis functions grows exponentially with dimension, leading many to suggest the use of adaptive nonlinear basis functions whose parameters can be determined by iterative methods. The author proposes a technique based on the idea that for most of the data, only a few dimensions of the input may be necessary to compute the desired output function. Additional input dimensions are incorporated only where needed. The learning procedure grows a tree whose structure depends upon the input data and the function to be approximated. This technique has a fast learning algorithm with no local minima once the network shape is fixed, and it can be used to reduce the number of required measurements in situations where there is a cost associated with sensing. Three examples are given: controlling the dynamics of a simulated planar two-joint robot arm, predicting the dynamics of the chaotic Mackey-Glass equation, and predicting pixel values in real images from pixel values above and to the left.