Event cameras with high dynamic range and temporal resolution, which are bio-inspired vision sensors, have shown great potential in event-based tracking tasks, particularly in scenarios involving rapid motion and low levels of illumination. Nonetheless, the efficient extraction of sparse information from event camera remains a persistent challenge. Meanwhile, the event camera works asynchronously, generating a continuous stream of events, rendering it highly compatible with Spiking Neural Networks (SNNs) due to their event-driven nature and low power consumption. Motivated by the issues mentioned above, we propose an Efficient Hybrid Cascade Tracker ( EHCT ) with SNN for object tracking in the event domain. We combine the transformer, convolutional network, and SNN structure skillfully to form the basic Hybrid CNN-SNN-Transformer (HCST) block structure, which is also the central component part of our EHCT network. The HCST block is primarily utilized to process the incoming event data and extract information from both local and global contexts. After several cascade HCST blocks, these two types of information will be efficiently integrated with the preprocessed raw data, which are then fed into the Classifier and Regressor head to produce the bounding box (bbox) of the tracked target. Extensive experiments on various event and RGB frame-based datasets demonstrated that our proposed EHCT algorithm outperforms most of the existing state-of-the-art trackers by a significant margin and also achieves a great advantage in terms of energy consumption. Our source code will be available at https://github.com/masac11/EHCT .

Spiking neural networks (SNNs), inspired by the inherent spiking computation paradigm of the biological neural systems, have exhibited superior energy efficiency in 2D classification tasks over traditional artificial neural networks (ANNs). However, the regression potential of SNNs has not been well explored, especially in 3D point cloud processing. In this paper, we propose noise-injected spiking graph convolutional networks to leverage the full regression potential of SNNs in 3D point cloud denoising. Specifically, we first emulate the noise-injected neuronal dynamics to build noise-injected spiking neurons. On this basis, we design noise-injected spiking graph convolution for promoting disturbance-aware spiking representation learning on 3D points. Starting from the spiking graph convolution, we build two SNN-based denoising networks. One is a purely spiking graph convolutional network, which achieves low accuracy loss compared with some ANN-based alternatives, while resulting in significantly reduced energy consumption on two benchmark datasets, PU-Net and PC-Net. The other is a hybrid architecture, which integrates some ANN-based learning operations and exhibits a high performance-efficiency trade-off with only a few time steps. Our work lights up SNN’s potential for 3D point cloud denoising, injecting new perspectives of exploring the deployment on neuromorphic chips while paving the way for developing energy-efficient 3D data acquisition devices.

The increasing accessibility and high security of electroencephalography (EEG) signals have led to growing attention in the field of biometrics. Compared to existing biometric methods, the performance of EEG-based is not satisfactory due to the complexity and instability of EEG signals. For these methods, on the one hand, directly using raw EEG time series makes it difficult to obtain deep features to reflect individual differences; On the other hand, there is a lack of effective feature extraction methods to explore the correlation between EEG spatiotemporal dimensions. To this end, this paper proposes a novel stochastic adaptive Fourier decomposition (SAFD) enhanced multi-level collaborative network, abbreviated as SAFD-MLCNet, for EEG Biometric Authentication. Firstly, SAFD can simultaneously represent the intrinsic characteristics of multiple signals, rendering it exceptionally suitable for processing multichannel EEG signals. As such, SAFD is used to derive EEG signals with enhanced correlated information for subsequent network inputs. Following this, the MLCNet combines a temporal-spatial convolution (TS-Conv) block and a temporal-spatial joint attention (TSJ-Attn) block is employed to extract multi-scale spatiotemporal features, with the TSJ-Attn focusing on the mutual interaction between local temporal and spatial features. Experimental studies have been performed on two benchmark databases for various tasks of internal attack scenarios, external attack scenarios, cross-session capabilities. The experimental results prove that the proposed method achieves a state-of-the-art performance on the EEG Biometric Authentication, especially on the EEG Motor Movement/Imagery Dataset containing 109 individuals, achieving a classification accuracy of 99. 39% and an equal error rate (EER) of 0. 29%.

We present a novel correspondence-learning model for real-time registration of partially overlapping point clouds, between which the relative translation is large and hence identifying the correspondences is challenging. Our goal is to improve the feature learning for accurate correspondence establishment, which enables the promotion of registration performance significantly in terms of efficiency. This is realized by two particular designs. The first is a graph-based feature extraction module, which aggregates both inter and intra contexts of the input point clouds simultaneously for strengthening the connection between inputs. The second is a feature refinement module, which uses sparse convolutions to further widen the feature differences of dissimilar structures. The two modules reinforce each other to improve correspondence learning for robust and fast point cloud registration with low overlap. We evaluate the method on both synthetic and real-world large-scale datasets. The results in real registration tasks show that our method attains competitive registration accuracy with state-of-the-art methods, and is almost two times faster than these competing methods in some scenarios.

Spiking neural networks (SNNs) have revolutionized neural learning and are making remarkable strides in image analysis and robot control tasks with ultra-low power consumption advantages. Inspired by this success, we investigate the application of spiking neural networks to 3D point cloud processing. We present a point-to-spike residual learning network for point cloud classification, which operates on points with binary spikes rather than floating-point numbers. Specifically, we first design a spatial-aware kernel point spiking neuron to relate spiking generation to point position in 3D space. On this basis, we then design a 3D spiking residual block for effective feature learning based on spike sequences. By stacking the 3D spiking residual blocks, we build the point-to-spike residual classification network, which achieves low computation cost and low accuracy loss on two benchmark datasets, ModelNet40 and ScanObjectNN. Moreover, the classifier strikes a good balance between classification accuracy and biological characteristics, allowing us to explore the deployment of 3D processing to neuromorphic chips for developing energy-efficient 3D robotic perception systems.

Abstract Spherical harmonics are a favorable technique for 3D representation, employing a frequency‐based approach through the spherical harmonic transform (SHT). Typically, SHT is performed using equiangular sampling grids. However, these grids are non‐uniform on spherical surfaces and exhibit local anisotropy, a common limitation in existing spherical harmonic decomposition methods. This paper proposes a 3D representation method using Fibonacci Spherical Harmonics (FSH3D). We introduce a spherical Fibonacci grid (SFG), which is more uniform than equiangular grids for SHT in the frequency domain. Our method employs analytical weights for SHT on SFG, effectively assigning sampling errors to spherical harmonic degrees higher than the recovered band‐limited function. This provides a novel solution for spherical harmonic transformation on non‐equiangular grids. The key advantages of our FSH3D method include: 1) With the same number of sampling points, SFG captures more features without bias compared to equiangular grids; 2) The root mean square error of 32‐degree spherical harmonic coefficients is reduced by approximately 34.6% for SFG compared to equiangular grids; and 3) FSH3D offers more stable frequency domain representations, especially for rotating functions. FSH3D enhances the stability of frequency domain representations under rotational transformations. Its application in 3D shape reconstruction and 3D shape classification results in more accurate and robust representations. Our code is publicly available at https://github.com/Miraclelzk/Fibonacci-Spherical-Harmonics .

Tracking and mapping functions in a monocular SLAM system remain active due to their challenging nature. In this paper, we propose a novel approach to perform the accurate and robust ego-motion estimation and provide the detail-preserving reconstruction in indoor environments. More specifically, we design a new algorithm called synchronous event measurement (SEM) to create event-based difference images (EDIs) so as to highlight frame-to-frame (F2F) difference. The observation indicates that F2F difference is highly correlated with the camera's motion change. We hereby feed EDIs into a deep convolutional neural network, in order to infer ego-motion of the camera. Subsequently, based on a monocular reconstruction framework (REMODE), we devise an algorithm named event region search or briefly ERS, to reduce possibility of mismatch on the depth estimation stage. Evaluations on a variety of datasets demonstrate the satisfactory performance of our proposed method: the ego-motion estimation is more accurate than some geometric based Visual Odometry (VO) and learning based approaches. The results are robust under extreme situations, such as brightness variation and motion blur. Meanwhile, our approach can provide more precise depth map with relatively rich textural information.

We propose improving the cross-target and cross-scene generalization of visual navigation through learning an agent that is guided by conceiving the next observations it expects to see. This is achieved by learning a variational Bayesian model, called NeoNav, which generates the next expected observations (NEO) conditioned on the current observations of the agent and the target view. Our generative model is learned through optimizing a variational objective encompassing two key designs. First, the latent distribution is conditioned on current observations and the target view, leading to a model-based, target-driven navigation. Second, the latent space is modeled with a Mixture of Gaussians conditioned on the current observation and the next best action. Our use of mixture-of-posteriors prior effectively alleviates the issue of over-regularized latent space, thus significantly boosting the model generalization for new targets and in novel scenes. Moreover, the NEO generation models the forward dynamics of agent-environment interaction, which improves the quality of approximate inference and hence benefits data efficiency. We have conducted extensive evaluations on both real-world and synthetic benchmarks, and show that our model consistently outperforms the state-of-the-art models in terms of success rate, data efficiency, and generalization.

3-D pose estimation for texture-less objects remains a challenging problem. Previous works either focus on a template matching method to find the nearest template as a candidate, or construct a Hough forest, which utilizes the offset of patches to vote for the object location and pose. By contrast, in this paper, we propose a comprehensive framework to directly regress 3-D poses for the candidates, in which a convolutional neural network-based triplet network is trained to extract discriminating features from the binary images. To make the features suitable for the regression task, a pose-guided method and a regression constraint are employed with the constructed triplet network. We show that the constraint reaches the goal of creating the correlation between the features and 3-D poses. Once the expected features are obtained, the object pose could be efficiently regressed, by training a regression network with a simple structure. For symmetric objects, depth images are treated as an additional channel to feed the triplet network. Experiments on the LineMOD and our own datasets demonstrate our method with high regression precision and efficiency.