DAVIS cameras, which output both event streams and frames simultaneously, are increasingly being used to address the primary object detection challenges posed by complex lighting and motion blur. Nevertheless, fully leveraging the abundant temporal information and effectively fusing data from these two modalities remains a formidable challenge. In this paper, we first design a multi-scale spatio-temporal aggregation (MSTA) module to distill richer semantic information from event frames. Secondly, we assimilate and harness the strengths of YOLOv8 and RT-DETR to develop an innovative encoder with Multi-scale Cross-modal dynamic Interactive fusion and multi-level feature interactive Fusion (MCIF). In MCIF, we propose a dynamic channel switching and spatial attention with learnable fusing factors (DCF-CSSA) to improve the complementary interaction of cross-modal features. Extensive experiments demonstrate that our approach (which we call SCNet) significantly outperforms existing state-of-the-art (SOTA) object detection methods that fuse events and frames, achieving an mAP50 improvement of 6.2% on PKU-DAVIS-SOD and 12% on DESC-MOD, both contain a large number of samples with challenging lighting conditions and motion blur.

Spiking neural networks (SNNs) have attracted much attention due to their low energy consumption and fast inference on neuromorphic hardware. Currently, the most effective way to implement deep SNNs is through ANN-SNN conversion, which combines the maturity of ANNs with the fast inference of SNNs, achieving accuracy comparable to ANNs on large-scale datasets. However, to achieve this accuracy, a large number of time steps are usually required, which undermines the low energy consumption and fast inference advantages of SNNs. When the number of time steps is very limited, the converted SNNs face a severe drop in accuracy, especially in the case of a single time step, which severely restricts the practical application of SNNs. In this paper, we analyze the main reasons why ANN- SNN cannot achieve lossless conversion with very limited number of time steps, and then proposes a group IF units with shared membrane potential to achieve lossless conversion of ANN-SNN at extremely low latency. We validate the effectiveness of our method on CIFAR-10, CIFAR-100, and ImageNet. Experiments show that our proposed method outperforms the existing state- of-the-art methods at the same number of time steps. Moreover, we have achieved an nearly lossless conversion of ANN-SNN in a single time step to get a high-accuracy SNN for the first time. For example, for VGG-16, our method only loses 0.8% and 3.15% of the accuracy compared to ANNs on CIFAR-10/100 in a single time step.

Event cameras, serving as innovative vision sensors, provide novel insights and approaches for image classification tasks. These cameras generate a sparse and discrete event stream, with each individual event carrying minimal information. Consequently, it becomes crucial to convert this event stream into a suitable event representation that aids in feature extraction and object recognition. In this research, we present a computationally efficient spatio-temporal event representation that not only preserves the spatio-temporal information of events in its entirety but also simplifies the computation of temporal information through linearly weighted timestamps. Furthermore, we propose an adaptive segmentation method for event streams. This method generates time bins that exhibit high robustness to motion speed by integrating both global and local distribution information of event counts. To verify the efficacy of our proposed method, we conducted experiments on three publicly available datasets. The results demonstrate that our method surpasses other methods on both N-Caltech101 and CIFAR10-DVS, with enhancements of 1.1% and 4.1% respectively, and produces competitive results on N-CARS.

Recommendation systems aim to provide personalized recommendations by analyzing user preferences. However, they encounter significant challenges of data sparsity and distribution skewness, which can impair their accuracy. To address these issues, we propose a collaborative recommendation approach integrating Knowledge Graph Enhancement and Contrastive Learning(KGECL). In this approach, the Knowledge Graph (KG) enhancement module first generates structurally diverse subgraphs via random perturbations and then calculates their structural stability scores to guide the generation of the user-item interaction(U-I) graph. The U-I graph enhancement module generates contrastive views from perturbed graphs using Variational Graph Autoencoders (VGAE)to alleviate the impact of skewed data distribution and sparsity. Finally, the enhanced KG is concatenated with the U-I graph to jointly construct embeddings for both users and items. Experimental results on three public datasets (MIND, yelp2018, and amazon-book) demonstrate that KGECL outperforms baseline models in terms of Recall and Normalized Discounted Cumulative Gain (NDCG), achieving the desired improvements.

It has become a great challenge to balance accuracy and diversity in recommendation systems. Graph Neural Networks (GNNs), while powerful, can lead to node representation homogeneity and information redundancy due to the indiscriminate aggregation of neighbor information. Therefore we propose a novel method that integrates maximum entropy neighbor selection with graph contrast learning to enhance the diversity of recommendations. The method introduces a strategy for neighbor selection based on maximum entropy to ensure a diverse subset of neighbors is chosen during the aggregation phase. A layer attention mechanism is implemented to address the over-smoothing issue, directing greater focus on higher-order neighbors. Furthermore, a loss re-weighting technique is applied to emphasize the learning of long-tail items. The overarching objective is to significantly improve recommendation diversity while maintaining system accuracy, underpinned by graph contrast learning method. Experimental results on the Beauty and MIND-small datasets demonstrate significant enhancements in the accuracy and diversity metrics of the proposed method. In particular, regarding the Recall@300 metric, a substantial improvement of up to 30.56% is observed. Conversely, the method experiences a mere 4.63% reduction in accuracy compared to the optimal baseline. This indicates that the proposed method markedly amplifies the diversity of recommendations without significantly compromising recommendation accuracy.

Video small object detection is a difficult task due to the lack of object information. Recent methods focus on adding more temporal information to obtain more potent high-level features, which often fail to specify the most vital information for small objects, resulting in insufficient or inappropriate features. Since information from frames at different positions contributes differently to small objects, it is not ideal to assume that using one universal method will extract proper features. We find that context information from the long-term frame and temporal information from the short-term frame are two useful cues for video small object detection. To fully utilize these two cues, we propose a long short-term feature enhancement network (LSTFE-Net) for video small object detection. First, we develop a plug-and-play spatiotemporal feature alignment module to create temporal correspondences between the short-term and current frames. Then, we propose a frame selection module to select the long-term frame that can provide the most additional context information. Finally, we propose a long short-term feature aggregation module to fuse long short-term features. Compared to other state-of-the-art methods, our LSTFE-Net achieves 4.4% absolute boosts in AP on the FL-Drones dataset. More details can be found at https://github.com/xiaojs18/LSTFE-Net.

As a new type of vision sensor with high temporal resolution, high dynamic range and low power consumption, event cameras are increasingly used in the field of computer vision. Since the output of an event camera is a sparse and discrete event stream and individual event contains very little information, it is crucial to convert the event stream into a suitable event representation to facilitate the extraction of features and the recognition of the objects. In order to construct clear and complete features from raw event data efficiently, this paper proposes a frame-based three-channel event representation method, in which the temporal channels take the timestamp of the latest event at each pixel as its value to avoid contour overlapping caused by object motion and summarizing timestamps, and an event-count channel is constructed based on the number of events at each pixel to retain the overall spatial distribution of object contours. An outlier processing is also proposed to avoid “feature disappearance” based on the statistical distribution of the event count. We validate our method on several public event-based datasets and compare it with the existing state-of-the-art event representation methods, experimental results show that our method achieves the best classification accuracy with a lower time complexity, and it also yields a high level of robustness to latency.

Although deep neural networks (DNNs) have led to many remarkable results in cognitive tasks, they are still far from catching up with human-level cognition in antinoise capability. New research indicates how brittle and susceptible current models are to small variations in data distribution. In this letter, we study the stochasticity-resistance character of biological neurons by simulating the input-output response process of a leaky integrate-and-fire (LIF) neuron model and proposed a novel activation function, rand softplus (RSP), to model the response process. In RSP, a scale factor [Formula: see text] is employed to mimic the stochasticity-adaptability of biological neurons, thereby enabling the antinoise capability of a DNN to be improved by the novel activation function. We validated the performance of RSP with a 19-layer residual network (ResNet) and a 19-layer visual geometry group (VGG) on facial expression recognition data sets and compared it with other popular activation functions, such as rectified linear units (ReLU), softplus, leaky ReLU (LReLU), exponential linear unit (ELU), and noisy softplus (NSP). The experimental results show that RSP is applied to VGG-19 or ResNet-19, and the average recognition accuracy under five different noise levels exceeds the other functions on both of the two facial expression data sets; in other words, RSP outperforms the other activation functions in noise resistance. Compared with the application in ResNet-19, the application of RSP in VGG-19 can improve a network's antinoise performance to a greater extent. In addition, RSP is easier to train compared to NSP because it has only one parameter to be calculated automatically according to the input data. Therefore, this work provides the deep learning community with a novel activation function that can better deal with overfitting problems.

Rainfall weather can always seriously deteriorate the quality of the outdoor monitoring system image. Since the decomposition based methods do not need to impose any restrictions on the types of rain, they have a wider application in removing the rain streaks. However, they still have the problems of rain residues in the low frequency component, and mis-matching the background and the rain streaks with the same gradient in the high frequency. In this condition, we propose an image rain removal algorithm based on the depth of field and sparse coding. The algorithm includes four steps: image decomposition, dictionary learning, atomic clustering based on Principal Component Analysis and Support Vector Machine, image revising based on the depth of field saliency map. Firstly, the image is decomposed by using the combination of bilateral filtering and short-time Fourier transform, so that the contour in the low-frequency part of the image can be better preserved. The depth of field saliency map of the image is utilized to eliminate the rain residues in the low frequency components, and also to solve the problem of mis-matching the background and the rain streaks with the same gradient in the high frequency components. The experimental results demonstrate that the proposed algorithm performs better both in rain removal and preserving the detailed information of the image than current methods.