Real-time decoding of neural activity is central to neuroscience and neurotechnology applications, from closed-loop experiments to brain-computer interfaces, where models are subject to strict latency constraints. Traditional methods, including simple recurrent neural networks, are fast and lightweight but often struggle to generalize to unseen data. In contrast, recent Transformer-based approaches leverage large-scale pretraining for strong generalization performance, but typically have much larger computational requirements and are not always suitable for low-resource or real-time settings. To address these shortcomings, we present POSSM, a novel hybrid architecture that combines individual spike tokenization via a cross-attention module with a recurrent state-space model (SSM) backbone to enable (1) fast and causal online prediction on neural activity and (2) efficient generalization to new sessions, individuals, and tasks through multi-dataset pretraining. We evaluate POSSM's decoding performance and inference speed on intracortical decoding of monkey motor tasks, and show that it extends to clinical applications, namely handwriting and speech decoding in human subjects. Notably, we demonstrate that pretraining on monkey motor-cortical recordings improves decoding performance on the human handwriting task, highlighting the exciting potential for cross-species transfer. In all of these tasks, we find that POSSM achieves decoding accuracy comparable to state-of-the-art Transformers, at a fraction of the inference cost (up to 9x faster on GPU). These results suggest that hybrid SSMs are a promising approach to bridging the gap between accuracy, inference speed, and generalization when training neural decoders for real-time, closed-loop applications.

During periods of quiescence, such as sleep, neural activity in many brain circuits resembles that observed during periods of task engagement. However, the precise conditions under which task-optimized networks can autonomously reactivate the same network states responsible for online behavior is poorly understood. In this study, we develop a mathematical framework that outlines sufficient conditions for the emergence of neural reactivation in circuits that encode features of smoothly varying stimuli. We demonstrate mathematically that noisy recurrent networks optimized to track environmental state variables using change-based sensory information naturally develop denoising dynamics, which, in the absence of input, cause the network to revisit state configurations observed during periods of online activity. We validate our findings using numerical experiments on two canonical neuroscience tasks: spatial position estimation based on self-motion cues, and head direction estimation based on angular velocity cues. Overall, our work provides theoretical support for modeling offline reactivation as an emergent consequence of task optimization in noisy neural circuits.

Estimation of depth from a single image is an important scene understanding task in computer vision. With the advent of Deep Learning and Convolutional Neural Networks, staggeringly high accuracies have been achieved in this task. With advancements in model optimization, it has been possible to deploy these models on edge devices, allowing for efficient depth estimation in safety-critical applications in robots, rovers, drones and even self-driving vehicles. However, these models are susceptible to attacks from malicious adversaries, which aim to distort the output of the model for a seemingly clean image by adding minute perturbations. In the real-world scenario, the most plausible attack is the adversarial patch, which can be printed and used as a physical adversarial attack against Deep Learning models. In the case of Monocular Depth Estimation, we show that small adversarial patches, which range from 0.7% to 5% of the image size, greatly worsen model performance. It is thus essential that these models are made robust using defense mechanisms, to defend against malicious inputs while also not reducing performance on clean images. Moreover, it is essential that the defense mechanism be computationally efficient, for real-time inference on edge devices. In this work, we propose the first defense mechanism against adversarial patches for a regression network, in the context of Monocular Depth Estimation on an edge device. The defense mechanism adds very little overhead time of 38 milliseconds on a Raspberry Pi 3 Model B, maintaining performance on clean images while also achieving near clean image levels of performance on adversarial inputs.

Psychological stress in human beings has been on a meteoric rise over the last few years. Chronic stress can have fatal consequences such as heart disease, cancer, suicide and so on. It is thus imperative to detect stress early on to prevent health risks. In this work, we discuss efficient and accurate stress and affect detection using scalable Deep Learning methods, that can be used to monitor stress real-time on resource-constrained devices such as low-cost wearables. By making inferences on-device, we solve the issues of high latency and lack of privacy which are prevalent in cloud-based computation. Using the concept of Early Stopping - Multiple Instance Learning, we build specialized models for stress and affect detection for 3 popular datasets in the domain, that have very low inference times but high accuracy. We introduce a metric ηcompto measure the computational savings from the use of these models. On average, our models show an absolute increase of 10% in overall accuracy over the benchmarks, computational savings of 95.39%, and an 18x reduction in inference times on a Raspberry Pi 3 Model B. This allows for efficient and accurate real-time monitoring of stress on low-cost resource-constrained devices.