This paper presents a novel approach to addressing the long-sequence problem in high-resolution medical images for Vision Transformers (ViTs). Using smaller patches as tokens can enhance ViT performance, but quadratically increases computation and memory requirements. Therefore, the common practice for applying ViTs to high-resolution images is either to: (a) employ complex sub-quadratic attention schemes or (b) use large to medium-sized patches and rely on additional mechanisms within the model to capture the spatial hierarchy of details. We propose Symmetrical Hierarchical Forest (SHF), a lightweight approach that adaptively patches the input image to increase token information density and encode hierarchical spatial structures into the input embedding. We then apply a reverse depatching scheme to the output embeddings of the transformer encoder, eliminating the need for convolution-based decoders. Unlike previous methods that modify attention mechanisms \wahib{or use a complex hierarchy of interacting models}, SHF can be retrofitted to any ViT model to allow it to learn the hierarchical structure of details in high-resolution images without requiring architectural changes. Experimental results demonstrate significant gains in computational efficiency and performance: on the PAIP WSI dataset, we achieved a 3$\sim$32$\times$ speedup or a 2.95\% to 7.03\% increase in accuracy (measured by Dice score) at a $64K^2$ resolution with the same computational budget, compared to state-of-the-art production models. On the 3D medical datasets BTCV and KiTS, training was 6$\times$ faster, with accuracy gains of 6.93\% and 5.9\%, respectively, compared to models without SHF.

Understanding the brain remains a major scientific challenge due to its complex structure and function. Unlike artificial neural networks, the biological brain features diverse biophysical properties essential to its function. Building an accurate digital replica using extensive anatomical and physiological data, which are available in standardized public databases, has emerged as a promising approach. In this study, a light-weight biophysical neuron simulator was developed and optimized to the supercomputer Fugaku. Good strong scaling was demonstrated in a benchmark model upto 152,064 compute nodes with 7.13 petaflops performance. In a more realistic scenario, a whole cerebral cortex of a mouse, consisting of 9 million biophysical neurons and 26 billion synapses, was simulated on the full-scale Fugaku with 145,728 nodes. These suggest that the present high-performance computing technology is ready to support the construction of a digital replica of the whole mammalian brain.

Attention-based models are proliferating in the space of image analytics, including segmentation. The standard method of feeding images to transformer encoders is to divide the images into patches and then feed the patches to the model as a linear sequence of tokens. For high-resolution images, e.g. microscopic pathology images, the quadratic compute and memory cost prohibits the use of an attention-based model, if we are to use smaller patch sizes that are favorable in segmentation. The solution is to either use custom complex multi-resolution models or approximate attention schemes. We take inspiration from Adapative Mesh Refinement (AMR) methods in HPC by adaptively patching the images, as a pre-processing step, based on the image details to reduce the number of patches being fed to the model, by orders of magnitude. This method has a negligible overhead, and works seamlessly with any attention-based model, i.e. it is a pre-processing step that can be adopted by any attention-based model without friction. We demonstrate superior segmentation quality over SoTA segmentation models for real-world pathology datasets while gaining a geomean speedup of $6.9 \times$ for resolutions up to $64 K^{2}$, on up to 2,048 GPUs.

The cerebellum plays an essential role in adaptive motor control. Once we are able to build a cerebellar model that runs in realtime, which means that a computer simulation of 1 s in the simulated world completes within 1 s in the real world, the cerebellar model could be used as a realtime adaptive neural controller for physical hardware such as humanoid robots. In this paper, we introduce "Realtime Cerebellum (RC)", a new implementation of our large-scale spiking network model of the cerebellum, which was originally built to study cerebellar mechanisms for simultaneous gain and timing control and acted as a general-purpose supervised learning machine of spatiotemporal information known as reservoir computing, on a graphics processing unit (GPU). Owing to the massive parallel computing capability of a GPU, RC runs in realtime, while reproducing qualitatively the same simulation results of the Pavlovian delay eyeblink conditioning with the previous version. RC is adopted as a realtime adaptive controller of a humanoid robot, which is instructed to learn a proper timing to swing a bat to hit a flying ball online. These results suggest that RC provides a means to apply the computational power of the cerebellum as a versatile supervised learning machine towards engineering applications.

In this paper, we present analog VLSI implementation of a resonate-and-fire neuron (RFN) model, and then consider noise effects on its performance of signal detection. The RFN circuit is a silicon spiking neuron that has second-order membrane dynamics and exhibits dynamic behavior, such as fast subthreshold oscillation, coincidence detection, frequency preference, and post-inhibitory rebound. Due to such dynamic behavior, the RFN circuit acts as a band-pass filter and a coincidence detector at circuit level. Through SPICE simulations, we will demonstrate noise effects on the coincidence detection and the frequency preference of the RFN circuit.

A network model of the entorhinal cortex layer II that consisted of stellate cells and an interneuron was constructed. The stellate cell reproduced subthreshold oscillations in the theta range as observed in physiological experiments. The subthreshold oscillations were recurrently synchronized through an inhibitory interneuron, and theta rhythm was generated. Anatomical evidences suggest that the layers II and V of the entorhinal cortex and the hippocampus may form a loop circuit with a delay time of about 25 ms for signals going around. With the loop circuit introduced to the present model, the stellate cells that represent spatial information fired successively at intervals of 25 ms, in depolarized phases of the theta cycles. The present entorhinal cortex model with the loop circuits reproduced the phase precession, which has been observed experimentally as a characteristic of place cells.