Vision-Language Models (VLMs) have shown remarkable capabilities in processing visual data through natural language, yet their potential in smart surveillance for abnormal event detection remains underexplored. Traditional anomaly detection systems rely on predefined event classes, limiting their ability to identify novel or ambiguous anomalies in diverse scenarios. To address this, we propose VEZA (Vision Encoding for Zero-shot Anomalies), a novel architecture lever-aging pre-trained VLMs for zero-shot anomaly detection. With VLMs, we integrate vision and language modalities to encode and caption video streams, enabling flexible and context-rich anomaly detection without surveillance-specific training. This dual-modality approach enhances robustness across varied real-world settings. Evaluated on the UCF -Crime dataset, VEZA achieved up to 89% in open-vocabulary anomaly detection and video-text retrieval tasks, with 45% R @ 1 on the Ubnormal dataset as an additional evaluation. These results demonstrate VEZA's ability to identify and retrieve a wide variety of abnormal events without the need for prior training on surveillance-specific scenarios.

Video facial expression recognition is useful for many applications and received much interest lately. Although some methods give good results in controlled environments (no occlusion), recognition in the presence of partial facial occlusion remains a challenging task. To handle facial occlusions, methods based on the reconstruction of the occluded part of the face have been proposed. These methods are mainly based on the texture or the geometry of the face. However, the similarity of the face movement between different persons doing the same expression seems to be a real asset for the reconstruction. In this paper we exploit this asset and propose a new method based on an auto-encoder with skip connections to reconstruct the occluded part of the face in the optical flow domain. To the best of our knowledge, this is the first work that directly reconstructs the movement for facial expression recognition. We validated our approach in the controlled CK+ datasets on which different occlusions were generated. Our experiments show that the proposed method reduces the gap in the recognition accuracy between occluded and unoccluded situations. We also compare our approach with existing state-of-the-art approaches. In order to lay the basis of a reproducible and fair comparison in the future, we also propose a new experimental protocol that includes occlusion generation and reconstruction evaluation.

Human action recognition in videos is an important issue in computer vision. We propose an approach based on the integration of partial or global human body segmentation in the classification process to deal with partial movements and immobility. Experimentation on UCF101 public dataset output competitive recognition accuracy related state of the art.

Face alignment remains difficult under uncontrolled conditions due to the many variations that may considerably impact facial appearance. Recently, video-based approaches have been proposed, which take advantage of temporal coherence to improve robustness. These new approaches suffer from limited temporal connectivity. We show that early, direct pixel connectivity enables the detection of local motion patterns and the learning of a hierarchy of motion features. We integrate local motion to the two predominant models in the literature, coordinate regression networks and heatmap regression networks, and combine it with late connectivity based on recurrent neural networks. The experimental results on two datasets, 300VW and SNaP-2DFe, show that local motion improves video-based face alignment and is complementary to late temporal information. Despite the simplicity of the proposed architectures, our best model provides competitive performance with more complex models from the literature.

In uncontrolled settings occlusions occur and interfere with facial expressions recognition task. It is interesting to limit the number of regions required for face expression recognition task in order to moderate the occlusion interference. We propose a weighting scheme that ranks the facial regions needed to recognize expressions. Weights are calculated based on the contribution of each region to boost recognition in presence of various occlusions. Intelligent facial frameworks, based on region ranks are computed in presence of static occlusions (such as glasses, hair, hand on the face). Evaluations conducted using motion information as the underlying descriptor show that our approach maintains, per expression, very good recognition rates under various static occlusions occurring in uncontrolled settings.

In this paper, we present a new approach to count the number of people that cross a counting line from monocular video images. The proposed approach accumulates image slices and estimates the optical flow on them. Then, it performs an online blob detection on these slices in order to extract the crossing persons. The number of persons associated to each blob is determined using a linear regression model applied to blob features which are the position, velocity, orientation and size. The proposed approach is validated on several datasets captured using either a vertical overhead or an oblique mounted camera. The real-time performance and the high counting accuracy of this approach in indoor and outdoor environments are also demonstrated.

In this paper, we present an effective method for human action recognition using statistical models based on optical flow orientations. We compute a distribution mixture over motion orientations at each spatial location of the video sequence. The set of estimated distributions constitutes the direction model, which is used as a mid-level feature for the video sequence. We recognize human actions using a distance metric to compare the direction model of a query sequence with the direction models of training sequences. The experimentations have been performed on standard datasets and have showed promising results.

In this technical demonstration, we present a tool that projects on a target scene the visual gaze of the people passing in front of it. The target scene could be a large plasma screen, an advertising poster, a shelf or a shop window. The projection of the visual gaze corresponds to the coordinates of a region that represents the person's location of interest in the target scene. It is an important problem with many applications that try to understand human behavior in a controlled environment such as in security or customized marketing. The visual gaze projection is influenced by the parameters of the camera, the settings of the target scene, the method used for the estimation of the gaze (e.g. the information about the head pose, the location of eyes centers and corners, or any combination of them), and the visual gaze starting point.

Video-surveillance systems are becoming more and more autonomous in the detection and the reporting of abnormal events. In this context, this paper presents an approach to detect abnormal situations in crowded scenes by analyzing the motion aspect instead of tracking subjects one by one. The proposed approach estimates sudden changes and abnormal motion variations of a set of points of interest (POI). The number of tracked POIs is reduced using a mask that corresponds to hot areas of the built motion heat map. The approach detects events where local motion variation is important compared to previous events. Optical flow techniques are used to extract information such as density, direction and velocity. To demonstrate the interest of the approach, we present the results on the detection of collapsing events in real videos of airport escalator exits.

We present a tool that automatically detects abnormal situations in crowded scenes in real time. The followed approach analyzes the general motion aspect, instead of tracking subjects one by one, by detecting abnormal optical flow patterns of tracked KLT points. The number of tracked points is reduced by using a learned mask. We define a measure that describes the situation abnormality based on crowd density, direction variance and distribution, mean velocity and sometimes trajectory matching. To demonstrate the interest of this approach, we present the results on the detection of collapsing events in real videos of airport escalator exits.

Eye movements are arguably the most natural and repetitive movement of a human being. The most mundane activity, such as watching television or reading a newspaper, involves this automatic activity which consists of shifting our gaze from one point to another. Identification of the components of eye movements (fixations and saccades) is an essential part in the analysis of visual behavior because these types of movements provide the basic elements used by further investigations of human vision. However, many of the algorithms that detect fixations present a number of problems. In this paper, we present the results of a new fixation identification technique that is based on clustering of eye positions, using projections and a projection aggregation applied to static pictures. We also present results of a new method that computes dispersion of eye fixations in videos considering a multi-user environment.

Eye movements are certainly the most natural and repetitive movement of a human being. The most mundane activity, such as watching television or reading a newspaper, involves this automatic activity which consists of shifting our gaze from one point to another. Identification of the components of eye movements (fixations and saccades) is an essential part in the analysis of visual behavior because these types of movements provide the basic elements used by further investigations of human vision. However, many of the algorithms that detect fixations present a number of problems. In this article, we present a new fixation identification technique that is based on clustering of eye positions, using projections and projection aggregation applied to static pictures. We also present a new method that computes dispersion of eye fixations in videos considering a multiuser environment. To demonstrate the performance and usefulness of our approach we discuss our experimental work with two different applications: on fixed image and video.

Our demo focuses on eye tracking on web, image and video data. We use some state-of-the-art measurements, such as scan path, to determine how the user sees web documents, images and videos. Our approach is characterised by automatic eye/gaze tracking with non intrusive sensors, mainly infrared cameras of web, image and video documents. We analyse eye/gaze tracking concerns spatial regions of static documents (images and web pages) and spatial zones of dynamic documents (video, sequence of web pages hyperlinked). In the context of dynamic documents, the eye/gaze tracking is processed image-per -image in video documents, and page-per-page in web documents. The result is more rough on video, and more accurate on images and hyperlinked web pages. Eye/gaze tracking on video is relatively new and unexplored in the literature.