Detecting anomalies in video surveillance, particularly for firearm detection, remains a critical challenge in public safety systems. Traditional methods often rely on human operators manually monitoring surveillance feeds, which is both inefficient and prone to error. Recent advances in deep learning (DL) offer a promising alternative by enabling models to identify anomalous events without relying on rare and difficult-to-obtain positive samples during training. In this paper, we propose an unsupervised firearm detection framework based on adversarial autoencoders (AAE) networks. By learning a robust representation of normal (negative) training data, the model is able to identify deviations indicative of firearm-related anomalies during inference. Anomaly scores for firearm detection are calculated using reconstruction errors, based on the probability that the test sample aligns with the prior distribution. Our approach enhances the interpretability of firearm related anomaly detection (AD) and demonstrates superior performance on benchmark firearm datasets. Experimental results demonstrate that the proposed method surpasses current state-of-the-art techniques by effectively identifying out-of-distribution (OOD) events in video frames, leveraging learned priors within the AAE architecture. Experimental results on benchmark firearm datasets, including VISILAB, UCF-Firearm, and YouTube demonstrate the effectiveness of our firearm detection approach, achieving an average precision (AP) of 95.2% and an average detection accuracy (ACC) of 95.6%.

ABSTRACT Deep learning (DL) has significantly transformed computer vision, demonstrating remarkable achievements and extensive real‐world applications. However, recent studies have highlighted a critical vulnerability of DL models to adversarial examples (AE), where slight perturbations in input data can lead to erroneous outputs. We observe that the behaviour of the AE is similar to a chaotic system, where a minor change in the input leads to a significantly different output. In response, we propose a novel approach for detecting and categorizing adversarial inputs encountered by classification neural networks. The proposed approach focuses on extracting statistical profiles, termed as chaotic feature vectors (CFVs), from a collection of features derived from ordinal patterns (OP). In this work, the proposed AE detection method is tested on seven attack methods and three image datasets including MNIST, FMNIST and CIFAR10. The results indicate that CFVs exhibit promising capabilities in discerning AE against various types of adversarial attacks on different datasets. This advancement lays the foundation for devising attack mitigation strategies, thereby enhancing the robustness and security of DL models in the face of adversarial threats.

Video surveillance systems can play a critical role in ensuring public safety by assisting in the early detection of potentially dangerous objects or actions, such as people carrying handguns or other weapons. Recent machine learning architectures based on multi-head self-attention modules have demonstrated their ability to process sequential data. In this work, we propose AWARE, a self-Attention based Weapon Activity Recognition architecture for video surveillance systems. The main idea behind this approach is to use a Transformer encoder module to extract relevant features from input video sequences and classify them as either weapon-related actions or non-dangerous actions. The input data is generated by combining 2D human pose keypoints and potential weapon locations generated by object detection models. We evaluate our proposed method on a new action recognition dataset composed of video sequences of gun-related actions. Experiments conducted show that the proposed method achieves better results than other similar methods in this context.

ABSTRACT Adversarial examples are an intriguing and critical topic in the field of machine learning. The impact of malignant perturbations on deep learning‐based systems, especially in safety‐critical applications, highlights a significant security concern. While most research has focused on artificially generated adversarial attacks–crafted through optimization algorithms and constrained perturbations, it is important to note that adversarial examples can also occur naturally, without any artificial manipulation, during the prediction of real‐world images. These naturally occurring adversarial examples pose unique challenges, as they are harder to detect and interpret. Despite their importance, the study of natural adversarial examples remains in its early stages. Fundamental questions remain unanswered: Do natural adversarial examples exhibit similar behaviours or properties as artificially generated ones? How should models be adapted to improve their robustness against such natural inputs? To address these questions, this work proposes an in‐depth analysis of activation maps to compare the internal behaviour of neural networks when processing clean images, artificially perturbed inputs and natural adversarial examples. A set of quantitative metrics is extracted from activation heatmaps at various network layers, including mean activation intensity, centroid displacement and standard reference image quality metrics. These measurements enable a systematic comparison of how the network attends to different image regions under varying conditions. The experimental results demonstrate that natural adversarial examples exhibit statistically significant differences in activation patterns compared to their artificial counterparts, suggesting that they may require distinct strategies for detection and defence.

Pediatric obesity can drastically heighten the risk of cardiometabolic alterations later in life, with insulin resistance standing as the cornerstone linking adiposity to the increased cardiovascular risk. Puberty has been pointed out as a critical stage after which obesity-associated insulin resistance is more difficult to revert. Timely prediction of insulin resistance in pediatric obesity is therefore vital for mitigating the risk of its associated comorbidities. The construction of effective and robust predictive systems for a complex health outcome like insulin resistance during the early stages of life demands the adoption of longitudinal designs for more causal inferences, and the integration of factors of varying nature involved in its onset. In this work, we propose an eXplainable Artificial Intelligence-based decision support pipeline for early diagnosis of insulin resistance in a longitudinal cohort of 90 children. For that, we leverage multi-omics (genomics and epigenomics) and clinical data from the pre-pubertal stage. Different data layers combinations, pre-processing techniques (missing values, feature selection, class imbalance, etc.), algorithms, training procedures were considered following good practices for Machine Learning. SHapley Additive exPlanations were provided for specialists to understand both the decision-making mechanisms of the system and the impact of the features on each automatic decision, an essential issue in high-risk areas such as this one where system decisions may affect people's lives. The system showed a relevant predictive ability (AUC and G-mean of 0.92). A deep exploration, both at the global and the local level, revealed promising biomarkers of insulin resistance in our population, highlighting classical markers, such as Body Mass Index z-score or leptin/adiponectin ratio, and novel ones such as methylation patterns of relevant genes, such as HDAC4, PTPRN2, MATN2, RASGRF1 and EBF1. Our findings highlight the importance of integrating multi-omics data and following eXplainable Artificial Intelligence trends when building decision support systems.

Abstract The phenomenon of adversarial examples is one of the most attractive topics in machine learning research these days. These are particular cases that are able to mislead neural networks, with critical consequences. For this reason, different approaches are considered to tackle the problem. On the one side, defense mechanisms, such as AutoEncoder-based methods, are able to learn from the distribution of adversarial perturbations to detect them. On the other side, chaos theory and Lyapunov exponents (LEs) have also been shown to be useful to characterize them. This work proposes the combination of both domains. The proposed method employs these exponents to add more information to the loss function that is used during an AutoEncoder training process. As a result, this method achieves a general improvement in adversarial examples detection performance for a wide variety of attack methods.

Early detection of the presence of dangerous objects such as handguns in Closed-Circuit Television (CCTV) images is vital to reduce the potential damage. In this work, a novel method for automatic detection of handguns in CCTV-like images based on a combination architecture which leverages body pose estimation is proposed. Weapon appearance features along with body pose features are combined to perform robust detection in typical surveillance environments where appearance features alone are not sufficient (e.g., because the handgun may appear too small or dark). Both CNN and recent transformer-based architectures are applied for visual feature extraction. Experiments on multiple datasets show that this approach improves state-of-the-art pose-based handgun detectors. An ablation study is also performed to verify the contribution of the pose processing branch and the false positive filter.

Soil-transmitted helminth infections are one of the most common healthcare problems worldwide and they especially affect to the poorest communities in tropical and subtropical areas. Nowadays, diagnosis of intestinal parasites is performed by highly skilled medical staff, directly examining samples in the laboratory via a microscope, a laborious and time-consuming work. Automatic deep learning-based object detection methods can help to automatically detect and identify intestinal parasitic eggs, or at least reduce the workload. In this work, the application of novel Transformer-based architectures is proposed to solve the parasitic egg detection task in microscopic images. Several detection methods and backbones have been analyzed and compared, obtaining up to 0.875 mIoU score on the dataset used for testing.

Automating the analysis of soil parameters can optimize the fertilization process, saving time and reducing the costs of food production, leading to a more sustainable agriculture. The work presented in this paper is part of the HYPERVIEW Challenge: Seeing Beyond the Visible. Several methods are proposed, based both on traditional approaches such as Support Vector Regression (SVR) and k-Nearest Neighbors (k-NN), as well as modern neural networks. A parameterized preprocessing stage has been proposed to deal with the varying size of the input data. The best results have been obtained with the k-NN model and the grid division of the images.

Automatic Non-rigid Histological Image Registration (ANHIR) challenge was organized to compare the performance of image registration algorithms on several kinds of microscopy histology images in a fair and independent manner. We have assembled 8 datasets, containing 355 images with 18 different stains, resulting in 481 image pairs to be registered. Registration accuracy was evaluated using manually placed landmarks. In total, 256 teams registered for the challenge, 10 submitted the results, and 6 participated in the workshop. Here, we present the results of 7 well-performing methods from the challenge together with 6 well-known existing methods. The best methods used coarse but robust initial alignment, followed by non-rigid registration, used multiresolution, and were carefully tuned for the data at hand. They outperformed off-the-shelf methods, mostly by being more robust. The best methods could successfully register over 98% of all landmarks and their mean landmark registration accuracy (TRE) was 0.44% of the image diagonal. The challenge remains open to submissions and all images are available for download.

Generalized nucleus segmentation techniques can contribute greatly to reducing the time to develop and validate visual biomarkers for new digital pathology datasets. We summarize the results of MoNuSeg 2018 Challenge whose objective was to develop generalizable nuclei segmentation techniques in digital pathology. The challenge was an official satellite event of the MICCAI 2018 conference in which 32 teams with more than 80 participants from geographically diverse institutes participated. Contestants were given a training set with 30 images from seven organs with annotations of 21,623 individual nuclei. A test dataset with 14 images taken from seven organs, including two organs that did not appear in the training set was released without annotations. Entries were evaluated based on average aggregated Jaccard index (AJI) on the test set to prioritize accurate instance segmentation as opposed to mere semantic segmentation. More than half the teams that completed the challenge outperformed a previous baseline. Among the trends observed that contributed to increased accuracy were the use of color normalization as well as heavy data augmentation. Additionally, fully convolutional networks inspired by variants of U-Net, FCN, and Mask-RCNN were popularly used, typically based on ResNet or VGG base architectures. Watershed segmentation on predicted semantic segmentation maps was a popular post-processing strategy. Several of the top techniques compared favorably to an individual human annotator and can be used with confidence for nuclear morphometrics.

While action recognition has become an important line of research in computer vision, the recognition of particular events such as aggressive behaviors, or fights, has been relatively less studied. These tasks may be extremely useful in several video surveillance scenarios such as psychiatric wards, prisons or even in personal camera smartphones. Their potential usability has led to a surge of interest in developing fight or violence detectors. One of the key aspects in this case is efficiency, that is, these methods should be computationally fast. "Handcrafted" spatiotemporal features that account for both motion and appearance information can achieve high accuracy rates, albeit the computational cost of extracting some of those features is still prohibitive for practical applications. The deep learning paradigm has been recently applied for the first time to this task too, in the form of a 3D Convolutional Neural Network that processes the whole video sequence as input. However, results in human perception of other's actions suggest that, in this specific task, motion features are crucial. This means that using the whole video as input may add both redundancy and noise in the learning process. In this work, we propose a hybrid "handcrafted/learned" feature framework which provides better accuracy than the previous feature learning method, with similar computational efficiency. The proposed method is compared to three related benchmark datasets. The method outperforms the different state-of-the-art methods in two of the three considered benchmark datasets.

This paper describes a specific tool for automatically segmenting and archiving of tissue microarray (TMA) cores in microscopy images at different magnifications. TMA enables researchers to extract the small cylinders of a single tissue (core sections) from histological sections and arrange them in an array on a paraffin block such that hundreds can be analyzed simultaneously. A crucial step to improve the speed and quality of this process is the correct localization of each tissue core in the array. However, usually the tissue cores are not aligned in the microarray, the TMA cores are incomplete and the images are noisy and with distorted colors. We develop a robust framework to handle core sections under these conditions. The algorithms are able to detect, stitch, and archive the TMA cores at different magnifications. Once the TMA cores are segmented they are stored in a relational database allowing their processing for further studies of benign-malignant classification. The method was shown to be reliable for handling the TMA cores and therefore enabling further large-scale molecular pathology research.

In this article we present an experience-based clinical decision support system (CDSS) for the diagnosis of Alzheimer's disease, which enables the discovery of new knowledge in the system and the generation of new rules that drive reasoning. In order to evolve an initial set of production rules given by medical experts we make use of the Set of Experience Knowledge Structure (SOEKS). An illustrative case of our system is also presented.

This paper describes a method based on an energy minimizing deformable model applied to the 3D biomechanical modeling of a set of organs considered as regions of interest (ROI) for radiotherapy. The initial model consists of a quadratic surface that is deformed to the exact contour of the ROI by means of the physical properties of a mass-spring system. The exact contour of each ROI is first obtained using a geodesic active contour model. The ROI is then parameterized by the vibration modes resulting from the deformation process. Once each structure has been defined, the method provides a 3D global model including the whole set of ROIs. This model allows one to describe statistically the most significant variations among its structures. Statistical ROI variations among a set of patients or through time can be analyzed. Experimental results are presented using the pelvic zone to simulate anatomical variations among structures and its application in radiotherapy treatment planning.

The widespread availability of portable computing power and inexpensive digital cameras are opening up new possibilities for retailers in some markets. One example is in optical shops, where a number of systems exist that facilitate eyeglasses selection. These systems are now more necessary as the market is saturated with an increasingly complex array of lenses, frames, coatings, tints, photochromic and polarizing treatments, etc. Research challenges encompass Computer Vision, Multimedia and Human-Computer Interaction. Cost factors are also of importance for widespread product acceptance. This paper describes a low-cost system that allows the user to visualize different glasses models in live video. The user can also move the glasses to adjust its position on the face. The system, which runs at 9.5 frames/s on general-purpose hardware, has a homeostatic module that keeps image parameters controlled. This is achieved by using a camera with motorized zoom, iris, white balance, etc. This feature can be specially useful in environments with changing illumination and shadows, like in an optical shop. The system also includes a face and eye detection module and a glasses management module.

This paper addresses the problem of describing the significant intra- and inter-variability of 3D deformable structures within 3D image data sets. In pursuing it, a 3D probabilistic physically based deformable model is defined. The statistically learned deformable model captures the spatial relationships between the different objects surfaces, together with their shape variations. The structures of interest in each volume are parameterized by the amplitudes of the vibration modes of a deformable spherical mesh. For a given 3D image in the training set, a vector containing the largest vibration modes describing the desired object is created. This random vector is statistically constrained by retaining the most significant variation modes of its Karhunen-Loeve (KL) expansion on the considered population. The surfaces of the modeled structures thus deform according to the variability observed in the training set. A preliminary application of a 3D multi-object model for the segmentation of 3D brain structures from MR images is presented.