Transformers significantly raise the performance limits across various tasks, spurring research into integrating them into spiking neural networks. However, a notable performance gap remains between existing spiking Transformers and their artificial neural network counterparts. Here, we first analyze the cause of this gap and attribute it to the dot product’s ineffectiveness in measuring similarity between spiking queries and keys, due to numerous non-spiking events. To address this, we propose a novel α-XNOR similarity measure tailored for spike trains. It redefines the correlation between non-spike pairs as a specific value α, effectively overcoming the limitations of dot-product similarity. Furthermore, considering the sparse nature of spike trains where spikes carry more information than non-spikes, the α-XNOR similarity correspondingly highlights the distinct importance of spikes over non-spikes. Extensive experiments demonstrate that α-XNOR similarity significantly improves performance across different spiking Transformer architectures on various static and neuromorphic datasets, further revealing the potential of spiking Transformers.

While design of high-performance lenses and image sensors has long been the focus of camera development, the size, weight, and power of image data processing components are currently the primary barriers to radical improvements in camera resolution. Here we show that deep learning--aided compressive sampling can reduce operating power on camera head electronics by 20 times or more. Traditional compressive sampling has to date been primarily applied in the physical sensor layer. We show here that with the aid of deep learning algorithms, compressive sampling is offers unique power management advantages in digital layer compression.