For video anomaly detection, it's both important to detect when the event happens and what the event is. The tasks of temporal grounding and semantic understanding can benefit from joint learning, but no existing work support it. To address this problem, we introduce VAGU (Video Anomaly Grounding and Understanding), the first benchmark designed to jointly evaluate semantic understanding and precise temporal grounding of anomalies, with comprehensive annotations and objective multiple-choice Video QA. Besides, we propose Glance then Scrutinize (GtS), the first training-free framework that achieves the best balance performance in both accuracy and efficiency. GtS uniquely balances high temporal precision and semantic interpretability while meeting practical speed requirements, outperforming previous methods in real-world scenarios. Furthermore, we introduce the JeAUG metric for holistic evaluation of both speed and accuracy. Extensive experiments demonstrate the superior effectiveness and practicality of our benchmark, framework, and metric.

Video anomaly detection (VAD) aims at detecting anomalous events in videos. Most existing VAD methods distinguish anomalies by learning visual features of the video, which usually face several challenges in real-world applications. First, these methods are mostly scene-dependent, whose performances degrade obviously once the scene changes. Second, these methods are incapable of giving explanations to the detected anomalies. Third, these methods cannot adjust definitions of normal or abnormal events during test time without retraining the model. One important reason for the drawbacks is that these visual-based methods mainly detect anomalies by fitting visual patterns rather than semantically understanding the events in videos. In this paper, we propose a training-free method named Semantic Understanding based Video Anomaly Detection (SUVAD) using multi-modal large language model (MLLM). By exploiting MLLMs to generate detailed texture descriptions for videos, SUVAD achieves semantic video understanding, and then detects anomalies directly by large language models. We also designed several techniques to mitigate the hallucination problem of MLLMs. Compared to the methods based on visual features, SUVAD obtains obviously better scene generalization, anomaly interpretability, and the ability of flexible adjustment of anomaly definitions. We evaluate our method on five mainstream datasets. The results show that SUVAD achieves the best performance among all the training-free methods.

Semi-supervised video anomaly detection (SVAD) is a challenging computer vision task due to the diversity, randomness, and rarity of abnormal events in videos. A series of important SVAD methods follow the frame prediction strategy, where the model is trained only on normal videos and thus frames with higher prediction error are regarded as anomalies. However, these methods often suffer from inadequate sensitivity to anomalies because the prediction error of anomalous frames is insufficiently distinguishable from the error of normal frames. In this paper, we propose a compact deformable 3D convolution (CD3D) for feature extraction in SVAD models, which effectively enhances the discriminating ability between normal and anomalous frames. In CD3D, the offsets of sampling locations are predicted by applying a set of extra separable 3D convolutions to multiple frames, so that the context information is better utilized with reduced computational costs. Our proposed method achieves the best performance among the methods without resorting to additional supervised information. It can also be conveniently applied to methods that utilize extra supervised information, which further enhances their performances.

In this paper, Multi-Task Linear Dependency Modeling is proposed to distinguish drug-related webpages that contain lots of images and text. Linear Dependency Modeling exploits semantic relations between images features and text features, and Multi-Task Learning takes advantage of metadata of webpages. Meaningful information of webpages can be made use of fully to improve classification accuracy. Experimental results show that Multi-Task Linear Dependency Modeling outperforms existing decision level and feature level combination methods and achieves the best performance.

Imaging spectropolarimetry is a new sensing method that can acquire the spectral, polarimetric, and spatial information of an interesting scene. They give the incomplete representations of a scene, respectively, and it is expected that combination of them will improve confidence in target identification and quality of the scene description. In this letter, a divide-and-conquer-based unsupervised spectropolarimetric data classification method is proposed to utilize the spatial, spectral, and polarimetric information jointly. First, a spectropolarimetric projection scheme is proposed to divide the whole data set into two parts: spatial-spectral and spatial-polarimetric domains. Then, a nonparametric technique is used to extract the homogeneous regions in these two domains. Each homogeneous region offers a reference spectrum and polarization, based on which a pseudosupervised spectropolarimetric classification scheme is developed by using evidence theory to fuse the information provided by the spectrum and polarization. The experimental results on real spectropolarimetric data demonstrate that the proposed divide-and-conquer-based classification scheme can achieve higher accuracy than the fuzzyc-means clustering method with spatial information constraints, which takes into account the spatial information during spectral and polarimetric clustering. Moreover, the experimental results also show the potential of spectropolarimetric classification.