Cognitive neural prostheses offer a potential therapy for neural pathway damage by modulating neural activities of downstream cortical regions. Previous studies using reinforcement learning (RL) frameworks have made significant progress in generating spike patterns that can effectively induce desired behaviors. However, these approaches have struggled to generate realistic neural activities that align with natural firing patterns. In this work, we propose a novel method that aligns the latent dynamics of transregional neural population activities through behavioral reinforcement, achieving both natural neural encoding and optimal behavioral performance. Specifically, we train two Transformer-based variational autoencoders (tVAEs) on upstream and downstream spike trains through self-supervised training. The latent dynamics inferred from the upstream tVAE encoder are aligned with the downstream latent dynamics. Functional spike patterns are then generated from the aligned latent representations via the downstream tVAE decoder, with the alignment weights tuned toward maximal behavioral rewards by policy gradient. We validate our method using neural recordings from the medial prefrontal cortex (mPFC) and the primary motor cortex (M1) of a Sprague Dawley rat performing a two-lever discrimination task. Preliminary results demonstrate that our method successfully generates neural activities that drive task success while preserving natural firing patterns at the neural population level. Moreover, our method outperforms previous RL-based models at both neural and behavioral levels, highlighting its potential in future real-time stimulation experiments with cognitive neural prostheses.

Brain-machine interface (BMI) technology has witnessed notable advancements, facilitating individuals with motor disabilities to effectively operate prosthetic limbs. Reinforcement learning (RL) has been employed within BMIs to train decoders that can interpret neural activity and translate it into movement intentions using reward information. Internal rewards, reflected in neural response to sensory feedback in the medial prefrontal cortex (mPFC), can be used for autonomous updates in RL-BMIs. Studies have shown that designed audio feedback improves subjects' learning abilities, while neural activity in the mPFC induced by audio feedback indicates future rewards information. These findings highlight the possibility to utilized neural modulation in mPFC upon audio-induced can serves intermediate guidance on decoder update feedback. However, the reliance on single neuron spike signals from mPFC has limitations and will be unavailable, especially in long-term BMI implants. Local field potentials (LFPs) provide neural ensemble information and have been proposed as an alternative long-term data source to overcome these limitations. This paper proposes to extract LFPs neuromodulations and relate them to audio-induced reward expectation information from mPFC neural activity by implementing a data-driven marked point process (MPP) methodology. We correlate synchronized spike activity to the transient events in the LFP broad high frequency (bhf) band (200-400Hz) in the mPFC of rats performing the two-lever press discrimination task. Compared with extracting LFP features from the binned spectrogram power, our approach improves 24.63% on average in the peak-signal-to-noise-ratio (PSNR) across subjects over our data segments. This study indicates that LFPs in the mPFC contains the information that can provide sensory-induced reward expectation information in long-term use and advances the development of autonomous guidance for BMI decoders.

Numerous studies suggest that learning related but different tasks prior to a new task makes it easier, possibly because of our brain's neural pattern alignment mechanism. Specifically, the neural patterns in the new task align with those in the learned task, enabling the reuse of knowledge from the previous task to aid learning in the new task. Brain-machine interface (BMI) is an excellent tool for analyzing the dynamics of neural population patterns during new task learning by directly recording neural signals from the brain. If we can repeat the process of aligning neural pattern using a point registration algorithm with the recorded neural signals, it would provide a computational tool to help us understand the brain mechanism during task learning. Additionally, the pre-trained decoder parameters from the old task can be reused to expedite learning in the new task. However, the existing Iterative Closest Point (ICP) method easily fails as it is sensitive to neural data distribution. This paper proposes a pair-wise Kullback Leibler (KL) divergence optimizing framework for stable neural pattern alignment. The KL divergence measures the difference between the data distribution of the previous task and the aligned new task. The alignment process is formulated as an optimization problem by minimizing the KL divergence. The proposed algorithm is tested in a simulated experiment where a rat learns a two-lever discrimination task from a one-lever pressing task. Three scenarios are designed to test the feasibility of our algorithm, including non-Gaussian neural pattern shapes, noisy neural data, and different alignment angles. The results demonstrate that the proposed method is more robust than ICP, indicating its potential to discover the brain's alignment mechanism more accurately.

This article proposes an adaptive control for a flexible manipulator (FM) under the influence of distributed disturbances, unknown dead zones, and input quantization. First, the hybrid effect of the unknown dead zone and input quantization is formulated and represented based on some essential transformations. Then, an adaptive robust quantized control with online updating laws is developed to address the uncertainty of the dead zone, ensure robustness and angle position, and dampen the vibration in the FM system. Subsequently, the Lyapunov theoretical analysis is employed to ensure the bounded stability of the system. Finally, numerical simulations and experiments with a Quanser platform are given to further verify the feasibility and superiority of the designed scheme.

Brain-machine interfaces (BMIs) help people to control external devices using the digital commands based on interpreted brain states from multi-scale neural data. Existing methods have extracted brain states using principal component analysis and clustering. However, they work on the spike count data, fail to exploit the full information of spike timings, and result in a relatively coarse time resolution. Quick-response BMIs call for fine temporal resolution brain states estimation. The high-temporal resolution can be achieved by directly operating on spike timings. A cross-intensity (CI) kernel family facilitate this by mathematically describing the spike timings in a functional reproducing kernel Hilbert space (RKHS). In this paper, we applied a CI kernel distance-based clustering method on multi-neuron spike trains for brain state estimation. We tested this method on the neural data from the primary motor cortex of a Sprague Dawley rat performing lever-press tasks. The spike trains are grouped into several clusters by proposed method. We found the timings of clustered brain states to be highly correlated to specific stages of lever-press process. We compared CI-based clustering with L1-norm based clustering on multi-neuron spike trains, and found that CI-based clustering on average has 24.3% improvement on the correlation with lever-press states. These clustered brain states can be used to correlate local field potentials, and this correlation potentially helps build long-term stable BMIs.

Motor brain machine interfaces (BMIs) interpret neural activities from motor-related cortical areas in the brain into movement commands to control a prosthesis. As the subject adapts to control the neural prosthesis, the medial prefrontal cortex (mPFC), upstream of the primary motor cortex (M1), is heavily involved in reward-guided motor learning. Thus, considering mPFC and M1 functionality within a hierarchical structure could potentially improve the effectiveness of BMI decoding while subjects are learning. The commonly used Kalman decoding method with only one simple state model may not be able to represent the multiple brain states that evolve over time as well as along the neural pathway. In addition, the performance of Kalman decoders degenerates in heavy-tailed nongaussian noise, which is usually generated due to the nonlinear neural system or influences of movement-related noise in online neural recording. In this letter, we propose a hierarchical model to represent the brain states from multiple cortical areas that evolve along the neural pathway. We then introduce correntropy theory into the hierarchical structure to address the heavy-tailed noise existing in neural recordings. We test the proposed algorithm on in vivo recordings collected from the mPFC and M1 of two rats when the subjects were learning to perform a lever-pressing task. Compared with the classic Kalman filter, our results demonstrate better movement decoding performance due to the hierarchical structure that integrates the past failed trial information over multisite recording and the combination with correntropy criterion to deal with noisy heavy-tailed neural recordings.

Reinforcement learning (RL)-based brain-machine interfaces (BMIs) obtain the mapping between neural activities and the subject's intention using reward. The advantage is to allow subjects to learn to control the external device without real limb movements. Internal-reward-based RL-BMIs train the decoder based on the reward information extracted from neural activities, which is a step towards autonomous BMI design. Studies have used medial prefrontal cortex (mPFC) activity to extract the internal reward when rodents are in the learning process. However, the reward and non-reward classification using single neuron spikes is noisy. In this paper, we explore the reward interpretation ability of local field potentials (LFPs) in the mPFC area of SD rats, especially on the high-frequency bands. We also investigate whether LFPs contain extra information over spikes by using support vector machine (SVM) as the classifier to distinguish the rewarding and non-rewarding trials. We find that among the three bands, namely, gamma (30-80Hz), high-gamma (80-200Hz) and bhfLFP (200-400Hz), the bhfLFP band has the highest decoding accuracy (86.97% for a high lever task and 79% for a low lever task). Compared with the spike only, the integrated LFP-spike feature has comparable or better classification performance. It potentially provides more stable internal reward for RL-based BMIs.

The medial prefrontal cortex (mPFC) and primary motor cortex (M1) are both actively involved in the reward guided learning. However, how the information is conveyed in spike trains between these two regions has not been investigated. Spike prediction models have been developed to predict the spike trains between two cortical areas, for example, CA3 to CA1 in hippocampus, as well as premotor to M1. In this paper, we investigate the co-activation between mPFC and M1 by comparing three spike prediction models with different nonlinear capacity. Our data was collected from the mPFC and M1 of a rat when it was learning a two-lever discrimination task. The mPFC spike trains are served as the input of models and M1 spike trains as the desired output. We compare the spike prediction performances of three models, including generalized linear model (GLM), second order GLM, and staged point-process model. Our results show that all three model outputs have similar discrete-time rescaling Kolmogorov-Smirnov test results and similar correlation coefficient (0.42 on average across neurons). The paired t-tests across neurons also show the lack of significant difference. The preliminary results indicate possible linear co-activation between the M1 and mPFC during the learning stage.

Modeling spike train transformation among brain regions helps in designing a cognitive neural prosthesis that restores lost cognitive functions. Various methods analyze the nonlinear dynamic spike train transformation between two cortical areas with low computational eficiency. The application of a real-time neural prosthesis requires computational eficiency, performance stability, and better interpretation of the neural firing patterns that modulate target spike generation. We propose the binless kernel machine in the point-process framework to describe nonlinear dynamic spike train transformations. Our approach embeds the binless kernel to eficiently capture the feedforward dynamics of spike trains and maps the input spike timings into reproducing kernel Hilbert space (RKHS). An inhomogeneous Bernoulli process is designed to combine with a kernel logistic regression that operates on the binless kernel to generate an output spike train as a point process. Weights of the proposed model are estimated by maximizing the log likelihood of output spike trains in RKHS, which allows a global-optimal solution. To reduce computational complexity, we design a streaming-based clustering algorithm to extract typical and important spike train features. The cluster centers and their weights enable the visualization of the important input spike train patterns that motivate or inhibit output neuron firing. We test the proposed model on both synthetic data and real spike train data recorded from the dorsal premotor cortex and the primary motor cortex of a monkey performing a center-out task. Performances are evaluated by discrete-time rescaling Kolmogorov-Smirnov tests. Our model outperforms the existing methods with higher stability regardless of weight initialization and demonstrates higher eficiency in analyzing neural patterns from spike timing with less historical input (50%). Meanwhile, the typical spike train patterns selected according to weights are validated to encode output spike from the spike train of single-input neuron and the interaction of two input neurons.

Brain-machine interfaces (BMIs) translate the neural activity into digital command to control external devices in accomplishing movement task, which could involve multiple stages of behaviors in sequence. Previous work generally discriminates the stage labels using a classifier and uses a combination of sub-decoders designed respectively for each stage. Without considering the time dynamics of neural activity, the classifier often introduces noisy estimation in stage prediction. Brain-controlling neuro-prosthesis requires the decoder to continuously output the kinematics interpretation on brain state for each time instance, including when to start or end each stage within the task smoothly and accurately. We propose to decode the kinematic states within stages and during stage transition in one paradigm. The kinematics state vector is extended, and the time dynamics is modelled in the state-observation framework. We validate our approach on a brain-controlled lever discrimination task. The rats need to adjust the neural activity to press the correct virtual lever and drives the brain state to trigger the start of the next trial. Compared with the existing method whose mean square error is 0.8038, our results show smoother transition prediction and better decoding accuracy with less mean square error which is 0.7922. And the correction rate of holding lever and rest is above 90%. These results help the subjects do the task with even less response time (2.81s) and shorter inter-trial duration (4.5s).

Neurons communicate nonlinearly through spike activities. Generalized linear models (GLMs) describe spike activities with a cascade of a linear combination across inputs, a static nonlinear function, and an inhomogeneous Bernoulli or Poisson process, or Cox process if a self-history term is considered. This structure considers the output nonlinearity in spike generation but excludes the nonlinear interaction among input neurons. Recent studies extend GLMs by modeling the interaction among input neurons with a quadratic function, which considers the interaction between every pair of input spikes. However, quadratic effects may not fully capture the nonlinear nature of input interaction. We therefore propose a staged point-process model to describe the nonlinear interaction among inputs using a few hidden units, which follows the idea of artificial neural networks. The output firing probability conditioned on inputs is formed as a cascade of two linear-nonlinear (a linear combination plus a static nonlinear function) stages and an inhomogeneous Bernoulli process. Parameters of this model are estimated by maximizing the log likelihood on output spike trains. Unlike the iterative reweighted least squares algorithm used in GLMs, where the performance is guaranteed by the concave condition, we propose a modified Levenberg-Marquardt (L-M) algorithm, which directly calculates the Hessian matrix of the log likelihood, for the nonlinear optimization in our model. The proposed model is tested on both synthetic data and real spike train data recorded from the dorsal premotor cortex and primary motor cortex of a monkey performing a center-out task. Performances are evaluated by discrete-time rescaled Kolmogorov-Smirnov tests, where our model statistically outperforms a GLM and its quadratic extension, with a higher goodness-of-fit in the prediction results. In addition, the staged point-process model describes nonlinear interaction among input neurons with fewer parameters than quadratic models, and the modified L-M algorithm also demonstrates fast convergence.

Reinforcement learning (RL)-based decoders in brain-machine interfaces (BMIs) interpret dynamic neural activity without patients' real limb movements. In conventional RL, the goal state is selected by the user or defined by the physics of the problem, and the decoder finds an optimal policy essentially by assigning credit over time, which is normally very time-consuming. However, BMI tasks require finding a good policy in very few trials, which impose a limit on the complexity of the tasks that can be learned before the animal quits. Therefore, this paper explores the possibility of letting the agent infer potential goals through actions over space with multiple objects, using the instantaneous reward to assign credit spatially. A previous method, attention-gated RL employs a multilayer perceptron trained with backpropagation, but it is prone to local minima entrapment. We propose a quantized attention-gated kernel RL (QAGKRL) to avoid the local minima adaptation in spatial credit assignment and sparsify the network topology. The experimental results show that the QAGKRL achieves higher successful rates and more stable performance, indicating its powerful decoding ability for more sophisticated BMI tasks as required in clinical applications.

Motor brain-machine interface (BMI) has great potentials in neural motor prostheses and has received increasing attention during the past decades in the neural engineering field. It requires an approach to decode neural activities that represents desired movements. Much of the progress in decoding algorithms has been driven by the availability of neural data, e.g. spike trains, in some research groups having animal laboratories and capable of performing surgery and building BMI systems. However, researchers in the neural signal processing field often face a dilemma of lacking neural data. To continue the innovation in decoding algorithms, this paper introduces a public neural dataset, the ZJU Neural Decoding Dataset (ZJUNDD). We give the detailed paradigm of the BMI system on monkey, including the experimental setup and the collection of 96-channel motor cortical activities. The dataset contains spike rates of neurons obtained by a consistent spike sorting method. To improve the data quality and reduce outliers, the spike data are carefully selected according to the quality of hand movements of the monkey. A standard protocol is provided for the assessment of decoding algorithms on the dataset, including the partition of training and testing sets, and the evaluation metrics. We also build an online evaluation system in order to enable a fair comparison between decoding approaches. Further, we benchmark several existing algorithms, which provides a basic performance of the methods. To the best of our knowledge, this is the first public dataset of spike trains for the decoding research of motor cortical activities.

Previous decoding algorithms for Brain Machine Interfaces (BMIs) reconstruct the kinematics from recorded activities of hundreds of neurons, which are not all related to the movement task. Decoding from all neurons not only brings problem towards model generalization but also a significant computation burden. Knowledge of neural receptive fields helps ascertain the neuron importance associate with the movements. We propose to apply information theoretical analysis based on an instantaneous tuning model to extract the candidate neuron subsets, which also reduces the computation complexity for the decoding process. The cortical distribution of extracted neuron subsets is analyzed and the statistical decoding performances using neuron subset selection are compared to the one by the full neuron ensemble.

Many decoding algorithms for brain machine interfaces' (BMIs) estimate hand movement from binned spike rates, which do not fully exploit the resolution contained in spike timing and may exclude rich neural dynamics from the modeling. More recently, an adaptive filtering method based on a Bayesian approach to reconstruct the neural state from the observed spike times has been proposed. However, it assumes and propagates a gaussian distributed state posterior density, which in general is too restrictive. We have also proposed a sequential Monte Carlo estimation methodology to reconstruct the kinematic states directly from the multichannel spike trains. This letter presents a systematic testing of this algorithm in a simulated neural spike train decoding experiment and then in BMI data. Compared to a point-process adaptive filtering algorithm with a linear observation model and a gaussian approximation (the counterpart for point processes of the Kalman filter), our sequential Monte Carlo estimation methodology exploits a detailed encoding model (tuning function) derived for each neuron from training data. However, this added complexity is translated into higher performance with real data. To deal with the intrinsic spike randomness in online modeling, several synthetic spike trains are generated from the intensity function estimated from the neurons and utilized as extra model inputs in an attempt to decrease the variance in the kinematic predictions. The performance of the sequential Monte Carlo estimation methodology augmented with this synthetic spike input provides improved reconstruction, which raises interesting questions and helps explain the overall modeling requirements better.

The previous decoding algorithms for brain machine interfaces are normally utilized to estimate animal's movement from binned spike rates, which loses spike timing resolution and may exclude rich neural dynamics due to single spikes. Based on recently proposed Monte Carlo sequential estimation algorithm on point process, we present a decoding framework to reconstruct the kinematic states directly from the multi-channel spike trains. Starting with analysis on the differences between the simulation and real BMI data, neural tuning properties are modeled to encode the movement information of the experimental primate as the pre-knowledge for Monte-Carlo sequential estimation for BMI. The preliminary kinematics reconstruction shows better results when compared with Kalman filter.

Adaptive filtering is normally utilized to estimate system states or outputs from continuous valued observations, and it is of limited use when the observations are discrete events. Recently a Bayesian approach to reconstruct the state from the discrete point observations has been proposed. However, it assumes the posterior density of the state given the observations is Gaussian distributed, which is in general restrictive. We propose a Monte Carlo sequential estimation methodology to estimate directly the posterior density. Sample observations are generated at each time to recursively evaluate the posterior density more accurately. The state estimation is obtained easily by collapse, i.e. by smoothing the posterior density with Gaussian kernels to estimate its mean. The algorithm is tested in a simulated neural spike train decoding experiment and reconstructs better the velocity when compared with point process adaptive filtering algorithm with the Gaussian assumption.

Linear or non-linear models are used in brain machine interfaces (BIMIs) to map the neural activity to the associated behavior, typically the primate's hand position. Linear models assume a linear relationship between neural activity and hand position that may not be the case. A solution would be time-delay neural network (TDNN) that provides effectively a nonlinear combination of linear models. However, this model results in a drastic increase of free parameters and slow convergence when trained by an error backpropagation learning rule. We propose to train the TDNN by scaled conjugate gradient, which avoids time-consuming linear search, coupled with weight decay to reduce the free parameters number and produce generally faster convergence.