Current electrophysiological approaches can track the activity of many neurons, yet it is usually unknown which cell-types or brain areas are being recorded without further molecular or histological analysis. Developing accurate and scalable algorithms for identifying the cell-type and brain region of recorded neurons is thus crucial for improving our understanding of neural computation. In this work, we develop a multimodal contrastive learning approach for neural data that can be fine-tuned for different downstream tasks, including inference of cell-type and brain location. We utilize multimodal contrastive learning to jointly embed the activity autocorrelations and extracellular waveforms of individual neurons. We demonstrate that our embedding approach, Neuronal Embeddings via MultimOdal Contrastive Learning (NEMO), paired with supervised fine-tuning, achieves state-of-the-art cell-type classification for two opto-tagged datasets and brain region classification for the public International Brain Laboratory Brain-wide Map dataset. Our method represents a promising step towards accurate cell-type and brain region classification from electrophysiological recordings.

Recent work has demonstrated that large-scale, multi-animal models are powerful tools for characterizing the relationship between neural activity and behavior. Current large-scale approaches, however, focus exclusively on either predicting neural activity from behavior (encoding) or predicting behavior from neural activity (decoding), limiting their ability to capture the bidirectional relationship between neural activity and behavior. To bridge this gap, we introduce a multimodal, multi-task model that enables simultaneous Neural Encoding and Decoding at Scale (NEDS). Central to our approach is a novel multi-task-masking strategy, which alternates between neural, behavioral, within-modality, and cross-modality masking. We pretrain our method on the International Brain Laboratory (IBL) repeated site dataset, which includes recordings from 83 animals performing the visual decision-making task. In comparison to other large-scale modeling approaches, we demonstrate that NEDS achieves state-of-the-art performance for both encoding and decoding when pretrained on multi-animal data and then fine-tuned on new animals. Surprisingly, NEDS’s learned embeddings exhibit emergent properties: even without explicit training, they are highly predictive of the brain regions in each recording. Altogether, our approach is a step towards a foundation model of the brain that enables seamless translation between neural activity and behavior.

Developing the deep learning (DL) technique is a promising way to improve renewable energy forecasting accuracy and offset the negative impacts of renewable energy on the power system. However, the application of the DL technique brings novel cyberthreats to the renewable energy forecast, and its cybersecurity has not received enough attention in previous literatures. To fill the gap, the vulnerability of renewable energy forecasting is, among the first, studied in-depth in this article. First, a novel cyberattack named adversarial learning attack (ALA) is proposed. The ALA is achieved by tampering with the meteorological data obtained by online weather forecasts from external application programming interfaces to undermine the renewable energy forecasting performance, which jeopardizes the power system operation. Then, an iterative algorithm is proposed to solve the ALA-based optimization problem. As the DL model is involved as optimization constraints, the optimization problem is nonconvex and NP-hard, which is unable to be solved by traditional approaches. The proposed algorithm utilizes the proximal gradient descent principle and is effective in iteratively exploring the near-optimal solution. At last, the impact of the ALA strategy on the power system operation is assessed, which considers the economic loss incurred and the potential hazards. The feasibility and efficacy of the ALA strategy are validated by conducting comprehensive and extensive experiments on the IEEE 30-bus benchmarks. The simulation results reveal that the ALA is able to impose severe economic losses on the operation and even induces disastrous hazards, such as power system collapse.