Rehabilitation benefits from a careful understanding of people's movements, but there are few widely used tools to facilitate this beyond a goniometer or, in special cases, a gait laboratory. Easy to use and accurate kinematic tracking would allow better characterization of patient's impairments, allow quantitatively tracking improvements in response to treatment, and support research for more effective therapies. Two rapidly evolving technologies are poised to make this more available: wearable inertial sensors and video based motion tracking, which provide complementary information. However, both present challenges ranging from ease of use to the clinical relevance of their outputs. Pose estimation frequently struggles with ambiguities from the image. Wearable sensors often require precise placement and calibration to produce meaningful outputs, and often suffer from drift. This work addresses these challenges with a pipeline that fuses stereo video with inertial data to produce temporally-smooth pose estimates. Biomechanical constraints during optimization permit mapping the estimated pose trajectories to joint angles. The fusion also finds the relative rotation of the wearable sensors on the subject, which allows estimating joint angles without video. The system is tested in the rehabilitation setting on a participant with spinal cord injury using a wheelchair and is able to track shoulder elevation, elevation plane, internal rotation, elbow flexion and wrist pronation both in the fused condition and from inertial sensors alone during wheelchair propulsion.

In recent years, artificial neural networks have achieved state-of-the-art performance for predicting the responses of neurons in the visual cortex to natural stimuli. However, they require a time consuming parameter optimization process for accurately modeling the tuning function of newly observed neurons, which prohibits many applications including real-time, closed-loop experiments. We overcome this limitation by formulating the problem as $K$-shot prediction to directly infer a neuron's tuning function from a small set of stimulus-response pairs using a Neural Process. This required us to developed a Factorized Neural Process, which embeds the observed set into a latent space partitioned into the receptive field location and the tuning function properties. We show on simulated responses that the predictions and reconstructed receptive fields from the Factorized Neural Process approach ground truth with increasing number of trials. Critically, the latent representation that summarizes the tuning function of a neuron is inferred in a quick, single forward pass through the network. Finally, we validate this approach on real neural data from visual cortex and find that the predictive accuracy is comparable to --- and for small $K$ even greater than --- optimization based approaches, while being substantially faster. We believe this novel deep learning systems identification framework will facilitate better real-time integration of artificial neural network modeling into neuroscience experiments.

Ambitious projects aim to record the activity of ever larger and denser neuronal populations in vivo. Correlations in neural activity measured in such recordings can reveal important aspects of neural circuit organization. However, estimating and interpreting large correlation matrices is statistically challenging. Estimation can be improved by regularization, i.e. by imposing a structure on the estimate. The amount of improvement depends on how closely the assumed structure represents dependencies in the data. Therefore, the selection of the most efficient correlation matrix estimator for a given neural circuit must be determined empirically. Importantly, the identity and structure of the most efficient estimator informs about the types of dominant dependencies governing the system. We sought statistically efficient estimators of neural correlation matrices in recordings from large, dense groups of cortical neurons. Using fast 3D random-access laser scanning microscopy of calcium signals, we recorded the activity of nearly every neuron in volumes 200 μm wide and 100 μm deep (150-350 cells) in mouse visual cortex. We hypothesized that in these densely sampled recordings, the correlation matrix should be best modeled as the combination of a sparse graph of pairwise partial correlations representing local interactions and a low-rank component representing common fluctuations and external inputs. Indeed, in cross-validation tests, the covariance matrix estimator with this structure consistently outperformed other regularized estimators. The sparse component of the estimate defined a graph of interactions. These interactions reflected the physical distances and orientation tuning properties of cells: The density of positive 'excitatory' interactions decreased rapidly with geometric distances and with differences in orientation preference whereas negative 'inhibitory' interactions were less selective. Because of its superior performance, this 'sparse+latent' estimator likely provides a more physiologically relevant representation of the functional connectivity in densely sampled recordings than the sample correlation matrix.