Spiking Neural Networks (SNNs) are increasingly explored for their energy efficiency and biological plausibility, offering a compelling alternative to conventional artificial neural networks in neuromorphic applications. However, even Spik-former a fully spiking adaptation of the Transformer model, can exhibit significant computational redundancy due to excessive spike activity, resulting in non-negligible energy consumption. In this paper, we introduce Masked Spikformer, a unified framework for regulating temporal spike activity in spiking Transformers through three complementary masking strategies: Random Spike Masking (RSM), Gaussian-Based Spike Masking (GSM), and Gaussian-Based Spike Weighting (GSW). These approaches encompass both binary masking and continuous, learnable temporal weighting. Our method is integrated into fully spiking architectures and applied consistently during both training and inference. Experimental results on neuromorphic benchmarks (CIFARIO-DVS and DVS128 Gesture) show that the proposed masking strategies significantly reduce energy consumption while maintaining high classification performance. Notably, the best accuracy on DVS128 Gesture is achieved by RSM with 70% masking, while GSW(i), the inverse Gaussian weighting variant, attains the highest accuracy on CIFARIO-DVS. RSM also provides the lowest energy consumption across both datasets, highlighting the effectiveness of temporal sparsity for energy-efficient spiking Transformers.

This paper provides a comprehensive review of the integration of Spiking Neural Networks (SNNs) and Transformers, combining the energy efficiency of SNNs with the high performance of Transformer architectures. By leveraging the event-driven nature of SNNs and the powerful self-attention mechanism of Transformers, this fusion aims to address the challenges of high energy consumption in deep learning while improving task accuracy, especially for complex datasets. We introduce the core concepts of SNNs and Transformers, reviewing state-of-the-art methods for their combination, including hybrid architectures. The performance of each architecture is presented thanks to both static and neuromorphic datasets, highlighting their advantages and limitations. This review also discusses the challenges of integrating self-attention into spiking architectures and outlines future research directions to further enhance model performance and energy efficiency.

Understanding the driving environment is one of the key factors in achieving an autonomous vehicle. In particular, the detection of anomalies in the traffic lane is a high priority scenario, as it directly involves vehicle's safety. Recent state of the art image processing techniques for anomaly detection are all based on deep learning of neural networks. These algorithms require a considerable amount of annotated data for training and test purposes. While many datasets exist in the field of autonomous road vehicles, such datasets are extremely rare in the railway domain. In this work, we present a new innovative dataset relevant for railway anomaly detection called RailSet. It consists of 6600 high-quality manually annotated images containing normal situations and 1100 images of railway defects such as hole anomaly and rails discontinuity. Due to the lack of anomaly samples in public images and difficulties to create anomalies in the railway environment, we generate artificially images of abnormal scenes, using a deep learning algorithm named StyleMapGAN. This dataset is created as a contribution to the development of autonomous trains able to perceive tracks damage in front of the train. The dataset is available at this link.

In the field of train transportation, personal injuries due to train automatic doors are still a common occurrence. This paper aims at implementing a computer vision solution as part of a safety detection system to identify automatic doors-related hazardous events to reduce their occurrence and their severity. Deep anomaly detection algorithms are often applied on CCTV video feeds to identify such hazardous events. However, the anomalous events identified by those algorithms are often simpler than most common occurrences in transport environments, hindering their widespread usage. Since such events are of quite a diverse nature and no dataset featuring them exist, we create a specilically-tailored dataset composed of real-case scenarios of hazardous events near train doors. We then study an anomaly detection algorithm from the literature on this dataset and propose a set of modifications to better adapt it to our railway context and to subsequently ease its application to a wider range of use-cases.

This paper introduces a novel approach for early recognition of human actions using 3D skeleton joints extracted from 3D depth data. We propose a novel, frame-by-frame and real-time descriptor called Body-part Directional Velocity (BDV) calculated by considering the algebraic velocity produced by different body-parts. A real-time Hidden Markov Models algorithm with Gaussian Mixture Models state-output distributions is used to carry out the classification. We show that our method outperforms various state-of-the-art skeleton-based human action recognition approaches on MSRAction3D and Florence3D datasets. We also proved the suitability of our approach for early human action recognition by deducing the decision from a partial analysis of the sequence.

In this paper, we present an approach for multicamera pedestrian detection exploiting the concepts of multiview geometry and the shapes of 3D geometric primitives. Multicamera occupancy maps provide peak responses corresponding to the object detection but suffer from several false detections known as ghosts. The novelty of this paper is the introduction of shape patterns which can model the objects, such as pedestrians, by defining a kernel function in the projected occupancy space. This kernel depends upon the geometry of the 3D primitives and also varies in relation to their position with respect to the cameras in the real world configuration. For multiple objects visible across several cameras, we define a formation model which is the convolution of this spatially varying kernel with the set of possible object locations. The locations corresponding to detections can thus be obtained through a deconvolution process. For efficient computations, we further propose an estimated deconvolution process specific to our kernel responses which can also be heavily parallelized. We show the application of this process towards pedestrian detection by studying various 3D cylindrical primitives. Experiments on two public dataset sequences, including comparison with another approach, show the efficiency of the proposed method in terms of pedestrian detection and ghost pruning, including in adverse and challenging conditions.

This paper describes an original joint obstacle detection and tracking method based on a Mean Shift algorithm and semi-dense disparity maps. The semi-dense disparity maps are computed with a local 1D fuzzy scanline stereo matching approach. Each map is associated to a confidence map that is used to remove bad matches. The Mean Shift algorithm is applied to simultaneously extract each vehicle and track the 3D points belonging to the same vehicle along the sequence. We show that several vehicles can be efficiently detected and that a semi-dense disparity map is sufficient to reach an accurate segmentation even when occlusions occur. This paper presents some results on real image sequences acquired in the context of Advanced Driver Assistance Systems.

We propose a bottom-up human detector that can deal with arbitrary poses and viewpoints. Heads, limbs and torsos are individually detected, and an efficient assembly strategy is used to perform the human detection and the part segmentation. Firstly, a topological model is used to represent the structure of the human body, and the topologically equivalent configurations are ranked with additional priors. Promising results prove the approach efficiency for detecting people in low-resolution and compressed images.

Automatic detection of a falling person in video sequences has interesting applications in video-surveillance and is an important part of future pervasive home monitoring systems. In this paper, we propose a multiview approach to achieve this goal, where motion is modeled using a layered hidden Markov model (LHMM). The posture classification is performed by a fusion unit, merging the decision provided by the independently processing cameras in a fuzzy logic context. In each view, the fall detection is optimized in a given plane by performing a metric image rectification, making it possible to extract simple and robust features, and being convenient for real-time purpose. A theoretical analysis of the chosen descriptor enables us to define the optimal camera placement for detecting people falling in unspecified situations, and we prove that two cameras are sufficient in practice. Regarding event detection, the LHMM offers a principle way for solving the inference problem. Moreover, the hierarchical architecture decouples the motion analysis into different temporal granularity levels, making the algorithm able to detect very sudden changes, and robust to low-level steps errors.

Energy-based methods for motion estimation in image sequences process the input data either in the spatiotemporal or in the frequency domain. In both cases, the algorithms already described in the literature often require a huge number of elementary operations. In this paper, we describe a class of velocity selective filters which yield an accurate detection of the edges moving in the sequence. We first present a filtering scheme based on a convolution operation computed on a finite size neighborhood and describe its properties in the spatiotemporal and frequency domains. Then, we show that filters with similar properties can be implemented recursively, i.e., as convolutions computed on infinite-size neighborhoods. As an example, we finally show the filters' responses in the case of two superimposed translational motions.

This paper is concerned with the use of belief theory to resolve the data association problem in the context of tracking and identifying people using audio and video data. In order to associate measurements with targets, the proposed method exploits different features such as color, position and acoustic parameters. This has the advantage of providing a robust solution to data association in challenging tracking scenarios.

When associating data in the context of multiple target tracking, one is faced with the problem of handling the target emergence and disappearance. In this paper we show that we are able to handle this issue using belief theory based data association method without the introduction of an additional hypothesis to the frame of discernment. Using a specific modelling of belief functions, this is done by detecting and managing a portion of a conflict, which originates from the non-exhaustivity of the frame of discernment. The proposed method is associative and does not rely on the order under which the beliefs are combined. We demonstrate the effectiveness of the proposed method with experiments on simulated data. Additionally, we compare it with the extended world based data association method where a virtual hypothesis is added to the frame of discernment.

Toward real-time implementation of optical flow methods, we present an IIR filter structure suitable for local motion analysis. The motion analysis framework is divided into three steps. A set of velocity tuned filters is first applied to the image sequence. Then, for each pixel, the filter which gives the maximal response is selected. Finally, a segmentation step allows a clustering of pixels according to the previously computed motion index. We only focus on the first step of the method. We present the structure of the proposed IIR filters and show the influence of the parameters on their selectivity.