We present Self-Organizing Visual Prototypes (SOP), a new training technique for unsupervised visual feature learning. Unlike existing prototypical self-supervised learning (SSL) methods that rely on a single prototype to encode all relevant features of a hidden cluster in the data, we propose the SOP strategy. In this strategy, a prototype is represented by many semantically similar representations, or support embeddings (SEs), each containing a complementary set of features that together better characterize their region in space and maximize training performance. We reaffirm the feasibility of non-parametric SSL by introducing novel non-parametric adaptations of two loss functions that implement the SOP strategy. Notably, we introduce the SOP Masked Image Modeling (SOP-MIM) task, where masked representations are reconstructed from the perspective of multiple non-parametric local SEs. We comprehensively evaluate the representations learned using the SOP strategy on a range of benchmarks, including retrieval, linear evaluation, fine-tuning, and object detection. Our pre-trained encoders achieve state-of-the-art performance on many retrieval benchmarks and demonstrate increasing performance gains with more complex encoders.

This paper introduces a novel approach to improving the training stability of self-supervised learning (SSL) methods by leveraging a non-parametric memory of seen concepts. The proposed method involves augmenting a neural network with a memory component to stochastically compare current image views with previously encountered concepts. Additionally, we introduce stochastic memory blocks to regularize training and enforce consistency between image views. We extensively benchmark our method on many vision tasks, such as linear probing, transfer learning, few-shot classification, and image retrieval on many datasets. The experimental results consolidate the effectiveness of the proposed approach in achieving stable SSL training without additional regularizers while learning highly transferable representations and requiring less computing time and resources.

Deep neural networks have consistently represented the state of the art in most computer vision problems. In these scenarios, larger and more complex models have demonstrated superior performance to smaller architectures, especially when trained with plenty of representative data. With the recent adoption of Vision Transformer (ViT) based architectures and advanced Convolutional Neural Networks (CNNs), the total number of parameters of leading backbone architectures increased from 62M parameters in 2012 with AlexNet to 7B parameters in 2024 with AIM-7B. Consequently, deploying such deep architectures faces challenges in environments with processing and runtime constraints, particularly in embedded systems. This paper covers the main model compression techniques applied for computer vision tasks, enabling modern models to be used in embedded systems. We present the characteristics of compression subareas, compare different approaches, and discuss how to choose the best technique and expected variations when analyzing it on various embedded devices. We also share codes to assist researchers and new practitioners in overcoming initial implementation challenges for each subarea and present trends for Model Compression.

The COVID-19 pandemic has impacted research directions, especially on biometrics. Unfortunately, contact fingerprinting is still widely used and can be a pitfall for spreading the virus disease to the public. On the good side, this pitfall renews the interest in user-acceptance contactless fingerprinting as an alternative to the old and widely used contact sensors. Due to the low contrast produced by touchless images, developing a reliable minutia extraction method remains a challenge. This work proposes and analyzes a residual squeeze-and-excitation U-shaped deep learning model for extracting minutiae in contactless fingerprint images. The results, evaluated on three public datasets, show that the proposed method is competitive against other minutia extraction algorithms and commercial tools. Our results show that the proposed method can achieve an improvement of up to 2.1 percentage points in the F1-score over the existing ones, which can lead to a decrease in the equal error rate of up to 1.42 percentage points in verification experiments.

Mandible segmentation from computed tomography (CT) and cone beam computed tomography (CBCT) images has several applications in craniomaxillofacial and oral surgery planning, as well as in implant dentistry. These applications include planning using virtual models, such as measurements and implant positioning, as well as physical models, such as the molding of plates for fracture correction on 3D-printed models before surgery. Traditional image segmentation algorithms do not yield satisfactory results because it is common for the mandibular teeth to be occluded with the maxillary teeth. Another region that presents the same problem is the condyle head with the temporal bone. In light of this, this paper investigates the problem of mandible segmentation from CT and CBCT images using a patch-based convolutional neural network. We also describe the creation of ground truth for our dataset from a triangular mesh.

According to the World Health Organization (2023), epilepsy is a chronic brain disorder that affects around 50 million people of all ages worldwide, mainly (approximately 80%) those living in low- and middle-income regions. The most common method used to diagnose epilepsy is Electroencephalography (EEG), which involves monitoring brain activity. However, an-alyzing EEG recordings to detect epileptic seizures can be a tiresome and time-consuming task, even for experts. In addition, different physicians may have varying levels of experience, leading to differing diagnostic opinions. As the main contribution of this work, we propose and evaluate a novel method that combines time-frequency transforms and 1D convolutional neural networks to extract and process EEG features. Experiments performed on a widely used public data set show that our approach is competitive when compared to other EEG signal classification methods.

Re-Identification has been widely leverage by tracking-by-detection Multi-Object Tracking methods to enhance the matching step. During training, Re-Identification is usually learned as a sub-task considering the projected centroid of the bounding boxes as the embedding tensor. This design is problematic for Re-Identification feature learning and leveraging because of the noise introduced by the big amount of occlusions. Specifically, the occlusion between objects of the target class causes embedding tensors to represent objects with wrong IDs and this specific scenario has been overlooked in previous literature. In this work we introduce an Adaptive use of Re-Identification features that aims to tackle this problem by using Re-Identification features only when they are reliable. Our method is generic and can be added to further boost performance to any Multi-Object Tracking method that uses Re-Identification. Results in multiple datasets using various base methods demonstrate the consistency of our method with state-of-the-art results.

Color plays a vital role in our perception and interaction with the world, particularly in the realm of video where its presence or absence holds significant importance. Consequently, the process of generating accurate representations of color in videos, where original color information is absent, is a crucial area of research. Despite recent attention to this subject, there remain several unresolved issues pertaining to enhancing result quality and exploring novel approaches. To address these gaps, we propose Swin Flow Video Colorization (SwinFVC), a framework that leverages state-of-the-art computer vision techniques, such as Swin Transformers, in conjunction with classical convolutional neural networks (CNNs) to extract color information from reference images. Additionally, we introduce the Flow Color encoding block, a subnetwork responsible for generating feature representations that effectively capture color dynamics across images and video streams. Our implementation was trained using the DAVIS and LDV datasets, and the results exhibited superior effectiveness compared to current state-of-the-art methods, as evidenced by the Fréchet Inception Distance (FID) and Color Distribution Consistency (CDC) metrics.

Two-dimensional (2D) cultures have long been the standard in vitro drug discovery paradigm. However, three-dimensional (3D) culture models, e.g. spheroids and organoids, has revolutionized the screening for new bioactive substances, drug repositioning and precision medicine, once they better mimic the natural cell environment, and consequently, provide more accurate information. Despite the advances, analyzing these images of 3D structures takes a lot of time, money, and effort. Therefore, better, more precise and more adaptive segmentation models are required due to the demand for analysis automation. In this work, by contrasting its performance with CNNs that concentrate on medical image segmentation, we explore the applicability of Transformer models to the human cancer spheroid task. Additionally, we release our own dataset that is prepared for semantic segmentation benchmarking. Our experiments showed that the CNN models were superior at handling training with fewer samples because the Transformer’s self-attention mechanism requires considerably more data and training time before converging to a learned pattern.

BACKGROUND: Although the development of sequencing technologies has provided a large number of protein sequences, the analysis of functions that each one plays is still difficult due to the efforts of laboratorial methods, making necessary the usage of computational methods to decrease this gap. As the main source of information available about proteins is their sequences, approaches that can use this information, such as classification based on the patterns of the amino acids and the inference based on sequence similarity using alignment tools, are able to predict a large collection of proteins. The methods available in the literature that use this type of feature can achieve good results, however, they present restrictions of protein length as input to their models. In this work, we present a new method, called TEMPROT, based on the fine-tuning and extraction of embeddings from an available architecture pre-trained on protein sequences. We also describe TEMPROT+, an ensemble between TEMPROT and BLASTp, a local alignment tool that analyzes sequence similarity, which improves the results of our former approach. RESULTS: The evaluation of our proposed classifiers with the literature approaches has been conducted on our dataset, which was derived from CAFA3 challenge database. Both TEMPROT and TEMPROT+ achieved competitive results on [Formula: see text], [Formula: see text], AuPRC and IAuPRC metrics on Biological Process (BP), Cellular Component (CC) and Molecular Function (MF) ontologies compared to state-of-the-art models, with the main results equal to 0.581, 0.692 and 0.662 of [Formula: see text] on BP, CC and MF, respectively. CONCLUSIONS: The comparison with the literature showed that our model presented competitive results compared the state-of-the-art approaches considering the amino acid sequence pattern recognition and homology analysis. Our model also presented improvements related to the input size that the model can use to train compared to the literature methods.

Recent advancements in autonomous vehicles and vehicular ad-hoc networks (VANETs) have presented diverse solutions for vehicle safety and automation. The demand to establish a connection between the two worlds has increased significantly to augment road safety and provide benefits to end users. Intelligent Transportation Systems (ITSs) vehicle station leaves a signed trace of its geographic location that is rated as personal data. An attacker can misuse existing V2V communication to track a vehicle’s CAM-trace and to avoid misusing CAMs to harm privacy, selectives communication approaches for CAMs should be chosen instead of continuous communication of current CAMs. In this article we propose a VANET topology learning methodology prioritizing anonymization that can use any existing Graph Learning framework. Further, this work enhances the quality of graph models applied to the context of VANETs and autonomous vehicles. With the real coordinates, establishment of a grid of cells and the generation and training of the graph, we can compare different frameworks generally used in these issues and one can chose the one that fits better to each scenario.

Biometric identification by contactless fingerprinting has been a trend in recent years, reinforced by the pandemic of the new coronavirus (COVID-19). Contactless acquisition tends to be a more hygienic acquisition category with greater user acceptance because it is less invasive and does not require the use of a surface touched by other people as traditional acquisition does. However, this area presents some challenging tasks. Contact-based sensors still generally provide greater biometric effectiveness since the minutiae are more pronounced due to the high contrast between ridges and valleys. On the other hand, contactless images typically have low contrast, so the methods fail with spurious or undetectable details, demonstrating the need for further studies in this area. In this work, we propose and analyze a robust scaled deep learning model for extracting minutiae in contactless fingerprint images. The results, evaluated on three datasets, show that the proposed method is competitive against other minutia extraction algorithms and commercial software.

RGB-D data is essential for solving many problems in computer vision . Hundreds of public RGB-D datasets containing various scenes, such as indoor, outdoor, aerial, driving, and medical, have been proposed. These datasets are useful for different applications and are fundamental for addressing classic computer vision tasks , such as monocular depth estimation. This paper reviewed and categorized image datasets that include depth information. We gathered 231 datasets that contain accessible data and grouped them into three categories: scene/objects, body, and medical. We also provided an overview of the different types of sensors, depth applications, and we examined trends and future directions of the usage and creation of datasets containing depth data, and how they can be applied to investigate the development of generalizable machine learning models in the monocular depth estimation field.

The success of Zero-Shot Action Recognition (ZSAR) methods is intrinsically related to the nature of semantic side information used to transfer knowledge, although this aspect has not been primarily investigated in the literature. This work introduces a new ZSAR method based on the relationships of actions-objects and actions-descriptive sentences. We demonstrate that representing all object classes using descriptive sentences generates an accurate object-action affinity estimation when a paraphrase estimation method is used as an embedder. We also show how to estimate probabilities over the set of action classes based only on a set of sentences without hard human labeling. In our method, the probabilities from these two global classifiers (i.e., which use features computed over the entire video) are combined, producing an efficient transfer knowledge model for action classification. Our results are state-of-the-art in the Kinetics-400 dataset and are competitive on UCF-101 under the ZSAR evaluation. Our code is available athttps://github.com/valterlej/objsentzsar

In the last few years, several works have tackled the problem of novel view synthesis from stereo images or even from a single picture. However, previous methods are computationally expensive, specially for high-resolution images. In this paper, we address the problem of generating a multiplane image (MPI) from a single high-resolution picture. We present the adaptive-MPI representation, which allows rendering novel views with low computational requirements. To this end, we propose an adaptive slicing algorithm that produces an MPI with a variable number of image planes. We present a new lightweight CNN for depth estimation, which is learned by knowledge distillation from a larger network. Occluded regions in the adaptive-MPI are inpainted also by a lightweight CNN. We show that our method is capable of producing high-quality predictions with one order of magnitude less parameters compared to previous approaches. The robustness of our method is evidenced on challenging pictures from the Internet.

Stereo vision is a growing topic in computer vision due to the innumerable opportunities and applications this technology offers for the development of modern solutions, such as virtual and augmented reality applications. To enhance the user's experience in three-dimensional virtual environments, the motion parallax estimation is a promising technique to achieve this objective. In this paper, we propose an algorithm for generating parallax motion effects from a single image, taking advantage of state-of-the-art instance segmentation and depth estimation approaches. This work also presents a comparison against such algorithms to investigate the trade-off between efficiency and quality of the parallax motion effects, taking into consideration a multi-task learning network capable of estimating instance segmentation and depth estimation at once. Experimental results and visual quality assessment indicate that the PyD-Net network (depth estimation) combined with Mask R-CNN or FBNet networks (instance segmentation) can produce parallax motion effects with good visual quality.

Person Re-Identification is a challenging task that aims to retrieve all instances of a query image across a system of non-overlapping cameras. Due to the various extreme changes of view, it is common that local regions that could be used to match people are suppressed, which leads to a scenario where approaches have to evaluate the similarity of images based on less informative regions. In this work, we introduce the Top-DB-Net, a method based on Top DropBlock that pushes the network to learn to focus on the scene foreground, with special emphasis on the most task-relevant regions and, at the same time, encodes low informative regions to provide high discriminability. The Top-DB-Net is composed of three streams: (i) a global stream encodes rich image information from a backbone, (ii) the Top DropBlock stream encourages the backbone to encode low informative regions with high discriminative features, and (iii) a regularization stream helps to deal with the noise created by the dropping process of the second stream, when testing the first two streams are used. Vast experiments on three challenging datasets show the capabilities of our approach against state-of-the-art methods. Qualitative results demonstrate that our method exhibits better activation maps focusing on reliable parts of the input images. The source code is available at: https://github.com/RQuispeC/top-dropblock.

Abstract Lung nodule classification is one of the main topics related to computer‐aided detection systems. Although convolutional neural networks (CNNs) have been demonstrated to perform well on many tasks, there are few explorations of their use for classifying lung nodules in chest X‐ray (CXR) images. In this work, we proposed and analyzed a pipeline for detecting lung nodules in CXR images that includes lung area segmentation, potential nodule localization, and nodule candidate classification. We presented a method for classifying nodule candidates with a CNN trained from the scratch. The effectiveness of our method relies on the selection of data augmentation parameters, the design of a specialized CNN architecture, the use of dropout regularization on the network, inclusive in convolutional layers, and addressing the lack of nodule samples compared to background samples balancing mini‐batches on each stochastic gradient descent iteration. All model selection decisions were taken using a CXR subset of the Lung Image Database Consortium and Image Database Resource Initiative dataset separately. Thus, we used all images with nodules in the Japanese Society of Radiological Technology dataset for evaluation. Our experiments showed that CNNs were capable of achieving competitive results when compared to state‐of‐the‐art methods. Our proposal obtained an area under the free‐response receiver operating characteristic curve of 7.76 considering 10 false positives per image (FPPI), and sensitivity values of 73.1% and 79.6% with 2 and 5 FPPI, respectively.

The theories about crime and correction have their inception in the eighteenth century, highly influenced by the anthropological thoughts emerging during the age of Enlightenment. Throughout the decades, the criminological studies observed their sociological essence encompassing practices from othe r scientific fields to explain the more contemporary questions, becoming Criminology an inherently interdisciplinary science as a result. The adoption of concepts from Exact Sciences is a recent moving, originating it a novel research area, called Computational Criminology, which employs procedures from Applied Mathematics, Statistics and Computer Science to provide original or enhanced solutions to such questions. One of the most prominent tasks brought by this rising field is crime prediction, which attempts to uncover potential targets for future police intervention and also help solving already committed offenses. The present comparative analysis thus investigates the employment of statistical inference by means of Bayesian network for predictive policing, using the openly accessible registers from Chicago Police Department. Numerous algorithms are available to learn the structure for a Bayesian network purely from data and a comparative examination about them is hence described, with the purpose to establish the most precise and efficient one, according to the attributes of the said criminal dataset, for the implementation of the intended inference.

Due to rapid advances in the development of surveillance cameras with high sampling rates, low cost, small size and high resolution, video-based action recognition systems have become more commonly used in various computer vision applications. Human operators can be supported with the aid of such systems to detect events of interest in video sequences, improving recognition results and reducing failure cases. In this work, we propose and evaluate a method to learn two-dimensional (2D) representations from video sequences based on an autoencoder framework. Spatial and temporal information is explored through a multi-stream convolutional neural network in the context of human action recognition. Experimental results on the challenging UCF101 and HMDB51 datasets demonstrate that our representation is capable of achieving competitive accuracy rates when compared to other approaches available in the literature.

Presentation attack detection is a challenging problem that aims at exposing an impostor user seeking to deceive the authentication system. In facial biometrics systems, this kind of attack is performed using a photograph, video, or 3D mask containing the biometric information of a genuine identity. In this paper, we propose a novel approach to detecting face presentation attacks based on intrinsic properties of the scene such as albedo, depth, and reflectance properties of the facial surfaces, which were recovered through a shape-from-shading (SfS) algorithm. To extract meaningful patterns from the different maps obtained with the SfS algorithm, we designed a novel shallow CNN architecture for learning features useful to the presentation attack detection (PAD). We performed several experiments considering the intra- and inter-dataset evaluation protocols. The obtained results showed the effectiveness of the proposed method considering several types of photo- and video-based presentation attacks, and in the cross-sensor scenario, besides achieving competitive results for the inter-dataset evaluation protocol.

A neural network is a mathematical model that is able to perform a task automatically or semi-automatically after learning the human knowledge that we provided. Moreover, a Convolutional Neural Network (CNN) is a type of neural network that has shown to efficiently learn tasks related to the area of image analysis, such as image segmentation, whose main purpose is to find regions or separable objects within an image. A more specific type of segmentation, called semantic segmentation, guarantees that each region has a semantic meaning by giving it a label or class. Since CNNs can automate the task of image semantic segmentation, they have been very useful for the medical area, applying them to the segmentation of organs or abnormalities (tumors). This work aims to improve the task of binary semantic segmentation of volumetric medical images acquired by Magnetic Resonance Imaging (MRI) using a preexisting Three-Dimensional Convolutional Neural Network (3D CNN) architecture. We propose a formulation of a loss function for training this 3D CNN, for improving pixel-wise segmentation results. This loss function is formulated based on the idea of adapting a similarity coefficient, used for measuring the spatial overlap between the prediction and ground truth, and then using it to train the network. As contribution, the developed approach achieved good performance in a context where the pixel classes are imbalanced. We show how the choice of the loss function for training can affect the final quality of the segmentation. We validate our proposal over two medical image semantic segmentation datasets and show comparisons in performance between the proposed loss function and other pre-existing loss functions used for binary semantic segmentation.

Nowadays, scene text detection has received a lot of attention due to its complexity given variations in terms of orientations, font size, aspect ratio, and natural backgrounds. In this vein, several deep neural networks have been proposed to deal with this challenging problem. However, such networks produce “heavy” models, hampering their use in applications running in devices with computational constraints. Additionally, few works are focused on the detection of multi-oriented and/or multi-lingual text. Herein, we propose an end-to-end tiny convolutional neural network for multi-oriented multi-lingual scene text called Pelee- Text. Experimental results show that Pelee-Text is at least 3 times smaller than its counterparts with a speed of 2.93 and 18.64 frames per second for its multi-scale and 768-scale versions, respectively. Moreover, in terms of F-measure, our method achieved competitive results on four well-known datasets, i.e., ICDAR'2011 (90.96%), ICDAR'2013 (85.24%), ICDAR'2015 (80.08%), and MSRA-TD500 (80.90%).

Emotion recognition based on multiple information channels has recently received significant attention. Some examples of multimodal emotion recognition applications include online learning, entertainment, biometric systems, human-computer interactions, interpersonal relations, and behavior prediction. Two natural and effective ways to express emotions in human-human interaction are speech and facial expressions. In this work, we develop and evaluate a hybrid deep convolutional neural network (CNN) to extract audio and visual features from videos. An audio signal is converted into an image representation as input to a 2D convolutional neural network, whereas visual information is extracted from parts of the face. We then fuse both audio and visual features, reducing them through Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA). Finally, K-Nearest Neighbor (K-NN), Support Vector Machine (SVM), Logistic Regression (LR) and Gaussian Naive Bayes (GNB) classifiers are employed for emotion recognition. Experiments are conducted on RML, eNTERFACE05 and BAUM-1s datasets. Results show that our model achieved competitive recognition rates compared to approaches available in the literature.

Human falls are more susceptible in advanced ages and the second most common cause of accidental death to elders, because of the later detecting falls became a crucial research topic to an aging population. To this effect, we propose and evaluate the employment of a multi-stream approach to detect fall events. Three features (optical flow, saliency map, and RGB data) fed to each stream of a VGG-16 and classified by an SVM of whether there was or not a fall event. Experiments are conducted on two datasets, URFD and FDD, achieving accuracy rates of 98.84% and 99.51%, respectively, which outperforms the majority of the reviewed solutions.

Recent deep learning techniques have achieved satisfactory results for various image-related problems. However, many research questions remain open in tasks involving video sequences. Several applications demand the understanding of complex events in videos, such as traffic monitoring, person re-identification, security and surveillance. In this work, we address the problem of human action recognition in videos through a multi-stream network that incorporates both spatial and temporal information. The main contribution of our work is a stream based on a new variant of the visual rhythm, called Learnable Visual Rhythm (LVR). We employ a deep network to extract features from the video frames in order to generate the rhythm. The features are collected at multiple depths of the network to enable the analysis of different abstraction levels. This strategy significantly outperforms the handcrafted version on the UCF101 and HMDB51 datasets. Experiments conducted on these datasets show that our final multi-stream network achieved competitive results compared to state-of-the-art approaches.

In this study, we explore some possibilities of using aerial images captured by Unmanned Aerial Vehicles (UAV) and discuss the benefits of using them in the context of intelligent agriculture. A novel method that supports the detection and segmentation of tree crowns, the delineation of shadows, and which shows the direction of sunlight is presented. It uses simple observation strategies and commonly used digital image processing techniques such as visual color enhancement and perception, morphological operations, and segmentation based on a region growing method. The proposal is evaluated using a dataset with different types of crop areas and pasture lands. The results indicate that the proposal can effectively deal with the detection and segmentation of elements of interest in the scene, as well as the indication of the right side of the light source.

Over the last years, recognition of emotions based on multimodal channels has received increasing attention from the scientific community. Many application fields can benefit from multimodal emotion recognition, such as human-computer interactions, educational software, behavior prediction, interpersonal relations. Speech and facial expressions are two natural and effective ways to express emotions in human-human interaction. In this work, we introduce a hybrid deep convolutional neural network to extract audio and visual features from videos. Initially, for extracting audio data, we transform the audio signal into an image representation as input to a 2D-Convolutional Neural Network (CNN). For extracting visual data, we introduce a Census-Transform (CT) based on CNN. Then, we fuse both audio and visual features, reducing them through Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA). Finally, K-Nearest Neighbor (K-NN), Support Vector Machine (SVM), Logistic Regression (LR) and Gaussian Naïve Bayes (GNB) classifiers are employed for emotion recognition. Experimental results on RML, eNTERFACE05 and BAUM-1s datasets demonstrated that our model reached competitive recognition rates compared to other state-of-the-art approaches.

As well as in other knowledge domains, deep learning techniques have revolutionized the development of image super-resolution approaches. State-of-the-art algorithms for this problem have employed convolutional neural networks in residual architectures with a number of layers and generic loss functions, such as L1 and Peak Signal-to-Noise Ratio (PSNR). These frameworks (architectures + loss functions) are generic and do not address the main characteristics of an image for human visual perception (luminance, contrast, and structure) resulting in better images, however, with noise mainly at the edges. In this work, we present an edge enhanced super-resolution (EESR) method using a novel residual neural network with focus on image edges and a mix of loss functions that use PSNR, L1, Multiple-Scale Structural Similarity (MS-SSIM), and a new loss function based on the pencil sketch technique. As main contribution, the proposed framework aims to leverage the limits of image super-resolution and presents an improvement of the results in terms of the SSIM metric and achieving competitive results for the PSNR metric.

A hand tracking method is presented in this work, which achieves the best results found in the literature for the public RWTH-BOSTON-50 dataset, with a tracking error rate of 8.5%. Its main contribution is the extraction of intrinsic features from RGB sign language movies. In order to avoid some common limitations of model and appearance-based tracking methods, a movement pattern analysis was used as a feature basis. Such feature can be succinctly described as a space-time graph of interest point movement similarity, which is arranged as trees of dense trajectory connections to track hands in RGB sign language movies. In addition to basic geometry operations, simple graph methods are employed in the process, making it effective for parallel processing of large movie datasets.

Suitable time series representations play an important role in classification tasks. In this letter, we investigate the use of recurrence-plot-(RP)-based representations in the classification of eucalyptus regions in remote sensing images. The proposed framework is composed of three steps. First, time series associated with image pixels are represented by RP images; next, RP images are characterized by means of visual description approaches; finally, we use a soft computing framework based on genetic programing to discover an effective combination of time series dissimilarity functions to combine extracted features. Performed experiments in a eucalyptus classification problem demonstrated that the proposed framework is effective when compared to approaches based on the use of time series itself.

Advances in digital technology have increased event recognition capabilities through the development of devices with high resolution, small physical dimensions and high sampling rates. The recognition of complex events in videos has several relevant applications, particularly due to the large availability of digital cameras in environments such as airports, banks, roads, among others. The large amount of data produced is the ideal scenario for the development of automatic methods based on deep learning. Despite the significant progress achieved through image-based deep networks, video understanding still faces challenges in modeling spatio-temporal relations. In this work, we address the problem of human action recognition in videos. A multi-stream network is our architecture of choice to incorporate temporal information, since it may benefit from pre-trained deep networks for images and from handcrafted features for initialization. Furthermore, its training cost is usually lower than video-based networks. We explore visual rhythm images since they encode longer-term information when compared to still frames and optical flow. We propose a novel method based on point tracking for deciding the best visual rhythm direction for each video. Experiments conducted on the challenging UCF101 and HMDB51 data sets indicate that our proposed stream improves network performance, achieving accuracy rates comparable to the state-of-the-art approaches.

A large amount of video content has been produced by compact and portable cameras. Several applications have been benefited from such growth of multimedia data, such as telemedicine, business conferencing, surveillance and security, entertainment, distance learning, and robotics. Video stabilisation is the process of detecting and removing undesired motion or instabilities from a video stream caused during the acquisition stage when handling the camera. In this work, the authors introduce and analyse a novel approach that identifies failures in the global motion estimation of the camera by means of local features. Moreover, they propose an optimisation method for computing a new estimate of the corrected motion. Experiments conducted on different video sequences are performed to demonstrate the effectiveness of the developed method. Results obtained with the stabilisation process are compared against the state‐of‐the‐art YouTube method.

First-person action recognition is a recent problem in computer vision, where an observer wears body cameras to understand and recognize actions from the captured video sequences. Technological advances have made it possible to offer small wearable cameras that can be attached onto bike helmets, belts, animal halters, among other accessories. Examples of potential applications include sports, security, healthcare, visual lifelogging, among others. In this paper, we propose a novel approach to first-person action recognition that consists in encoding video appearance, shape and motion information as visual rhythms and describing them through texture analysis. Experiments are conducted on the DogCentric Activity and JPL First-Person Interaction datasets, showing accuracy improvement over the baselines.

The recognition of pollen grains is a challenging task since they are three-dimensional structures with complex morphological characteristics. Palynologists are responsible for studying pollen, spores and similar microscopic plant structures. In this work, we develop and analyze an automatic method for classification of pollen grain images based on a set of features and classifiers. Predictions of different classifiers are fused into an ensemble rule of majority voting. Experiments conducted on two datasets containing different types of pollen grains are used to demonstrate the effectiveness of the proposed approach.

Video summarization plays an important role in providing a compact representation of large volumes of video sequences through the search for the most informative or representative portions of their content. In this work, we propose and analyze a novel video summarization method based on texture and color information to describe the video frames in order to produce an effective visual abstract of the video sequence. Experiments conducted on a data set containing distinct genres demonstrate that the proposed method is capable of generating competitive video summaries when compared to other approaches available in the literature.

Gender recognition from face images is a challenging problem with applications in various knowledge domains, such as biometrics, security and surveillance, human-computer interaction, among others. In this work, we propose and evaluate a novel method for gender recognition based on a geometric descriptor constructed from a pre-defined face shape model. The proposed approach, tested on four different face datasets, achieved superior results for most cases when compared to other methods based on geometric descriptors.

Despite its associated challenges, the development of mechanisms for storing, indexing, transmitting and visualizing multimedia content is crucial in order to efficiently deal with the large amount of data generated by several different sources. Digital video technology has advanced rapidly, such that temporal video segmentation methods for automatically detecting transitions in video sequences play an important role in the content analysis tasks. In this work, we propose and evaluate a video shot boundary detection approach based on the Weber local descriptor. Experiments conducted on different datasets demonstrate the effectiveness of our method, whose results are compared against other approaches of the literature.

Emotion recognition based on facial expressions plays an important role in numerous applications, such as affective computing, behavior prediction, human-computer interactions, psychological health services, interpersonal relations, and social monitoring. In this work, we describe and analyze an emotion recognition system based on facial expressions robust to occlusions through Census Transform Histogram (CENTRIST) features. Initially, occluded facial regions are reconstructed by applying Robust Principal Component Analysis (RPCA). CENTRIST features are extracted from the facial expression representation, as well as Local Binary Patterns (LBP), Local Gradient Coding (LGC) and an extended Local Gradient Coding (LGC-HD). Then, the feature vector is reduced through Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA). For facial expression recognition, K-nearest neighbor (K-NN) and Support Vector Machine (SVM) classifiers are applied and tested. Experimental results on two public data sets demonstrated that the CENTRIST representation achieved competitive accuracy rates for occluded and non-occluded facial expressions compared to other state-of-the-art approaches available in the literature.

In recent years, we can observe an increasing use of biometric technology in our daily lives. Face recognition has several advantages over other biometric modalities, since that it is natural, nonintrusive, and it is a task that humans perform routinely and effortlessly. Following a recent trend in this research field, this paper focuses on a part-based face recognition, exploring and evaluating specific descriptions / classifications for each facial part. Experimental results obtained in three public datasets (AR Face, MUCT and XM2VTS), assessing 15 approaches in each facial part and two score fusion strategies, show that features and classifiers specific for facial part can improve the accuracy of biometric systems, achieving error rates close to zero in some cases, including scenarios where the false acceptance cases are critical.

Due to the large amount of videos generated through several data sources, the development of efficient mechanisms for storing, indexing, retrieving and visualizing their content is a challenging task. Temporal video segmentation is the automatic process of detecting transitions in video sequences, which is a fundamental step in the analysis of video content. This work proposes and evaluates an improved shot detection method based on the fusion of multiple frame dissimilarity measures and an adaptive threshold strategy. Experimental results demonstrate that the combination of different temporal features associated with an adequate threshold estimation can substantially improve the performance of individual methods.

Spatial resolution enhancement of license plate images in real scenarios plays an important role in the fields of criminal investigation and forensic science. This paper presents a learning-based single-image super-resolution method that uses a priori knowledge of the input as the plate images captured at poor quality and very low resolution. The proposed method employs a decision tree to classify the input image and the classification results are used to weight the image patches in the reconstruction step. Additionally, a histogram equalization is performed to improve the performance of the classifier. Experiments conducted on synthetic and real-world images demonstrate that the proposed method is capable of producing satisfactory results.

In this paper, we explore transformed spaces, represented by image illuminant maps, to propose a methodology for selecting complementary forms of characterizing visual properties for an effective and automated detection of image forgeries. We combine statistical telltales provided by different image descriptors that explore color, shape, and texture features. We focus on detecting image forgeries containing people and present a method for locating the forgery, specifically, the face of a person in an image. Experiments performed on three different open-access data sets show the potential of the proposed method for pinpointing image forgeries containing people. In the two first data sets (DSO-1 and DSI-1), the proposed method achieved a classification accuracy of 94% and 84%, respectively, a remarkable improvement when compared with the state-of-the-art methods. Finally, when evaluating the third data set comprising questioned images downloaded from the Internet, we also present a detailed analysis of target images.

This paper presents a novel Hybrid Genetic Algorithm (HGA) for image denoising, whose main purpose is to restore images while preserving relevant information, for instance, texture and edges. The proposed method combines operators available in existing evolutionary methods, such as crossover, mutation and population reinitialization with some state-of-the-art image denoising methods. Experiments are conducted on a set of noise contaminated images commonly used by the scientific community as benchmark, where different levels of noise are applied to the images. The results achieved by the proposed method are compared against image denoising methods. The HGA performance demonstrated to be very effective and competitive, outperforming other approaches in several levels of noise.

Biometrics systems have significantly improved person identification and authentication, playing an important role in personal, national, and global security. However, these systems might be deceived (or spoofed) and, despite the recent advances in spoofing detection, current solutions often rely on domain knowledge, specific biometric reading systems, and attack types. We assume a very limited knowledge about biometric spoofing at the sensor to derive outstanding spoofing detection systems for iris, face, and fingerprint modalities based on two deep learning approaches. The first approach consists of learning suitable convolutional network architectures for each domain, whereas the second approach focuses on learning the weights of the network via back propagation. We consider nine biometric spoofing benchmarks - each one containing real and fake samples of a given biometric modality and attack type - and learn deep representations for each benchmark by combining and contrasting the two learning approaches. This strategy not only provides better comprehension of how these approaches interplay, but also creates systems that exceed the best known results in eight out of the nine benchmarks. The results strongly indicate that spoofing detection systems based on convolutional networks can be robust to attacks already known and possibly adapted, with little effort, to image-based attacks that are yet to come.

Spoofing attacks or impersonation can be easily accomplished in a facial biometric system wherein users without access privileges attempt to authenticate themselves as valid users, in which an impostor needs only a photograph or a video with facial information of a legitimate user. Even with recent advances in biometrics, information forensics and security, vulnerability of facial biometric systems against spoofing attacks is still an open problem. Even though several methods have been proposed for photo-based spoofing attack detection, attacks performed with videos have been vastly overlooked, which hinders the use of the facial biometric systems in modern applications. In this paper, we present an algorithm for video-based spoofing attack detection through the analysis of global information which is invariant to content, since we discard video contents and analyze content-independent noise signatures present in the video related to the unique acquisition processes. Our approach takes advantage of noise signatures generated by the recaptured video to distinguish between fake and valid access videos. For that, we use the Fourier spectrum followed by the computation of video visual rhythms and the extraction of different characterization methods. For evaluation, we consider the novel unicamp video-attack database, which comprises 17 076 videos composed of real access and spoofing attack videos. In addition, we evaluate the proposed method using the replay-attack database, which contains photo-based and video-based face spoofing attacks.

Despite important recent advances, the vulnerability of biometric systems to spoofing attacks is still an open problem. Spoof attacks occur when impostor users present synthetic biometric samples of a valid user to the biometric system seeking to deceive it. Considering the case of face biometrics, a spoofing attack consists in presenting a fake sample (e.g., photograph, digital video, or even a 3D mask) to the acquisition sensor with the facial information of a valid user. In this paper, we introduce a low cost and software-based method for detecting spoofing attempts in face recognition systems. Our hypothesis is that during acquisition, there will be inevitable artifacts left behind in the recaptured biometric samples allowing us to create a discriminative signature of the video generated by the biometric sensor. To characterize these artifacts, we extract time-spectral feature descriptors from the video, which can be understood as a low-level feature descriptor that gathers temporal and spectral information across the biometric sample and use the visual codebook concept to find mid-level feature descriptors computed from the low-level ones. Such descriptors are more robust for detecting several kinds of attacks than the low-level ones. The experimental results show the effectiveness of the proposed method for detecting different types of attacks in a variety of scenarios and data sets, including photos, videos, and 3D masks.

Automatic data classification is a computationally intensive task that presents variable precision and is considerably sensitive to the classifier configuration and to data representation, particularly for evolving data sets. Some of these issues can best be handled by methods that support users' control over the classification steps. In this paper, we propose a visual data classification methodology that supports users in tasks related to categorization such as training set selection; model creation, application and verification; and classifier tuning. The approach is then well suited for incremental classification, present in many applications with evolving data sets. Data set visualization is accomplished by means of point placement strategies, and we exemplify the method through multidimensional projections and Neighbor Joining trees. The same methodology can be employed by a user who wishes to create his or her own ground truth (or perspective) from a previously unlabeled data set. We validate the methodology through its application to categorization scenarios of image and text data sets, involving the creation, application, verification, and adjustment of classification models.

Although several methods for action recognition have been proposed in the literature, many of them have limitations in terms of applicability in real-life situations. Despite satisfactory accuracy rates achieved by a number of methods, an effective action recognition system requires workability in real time. However, this feature usually comes along with certain loss in accuracy. In this paper, we present a real-time action recognition method that achieves state-of-the-art accuracy. By accumulating shape information over a sliding window on the video frames, the method extracts and processes silhouettes with little computational effort. Simple descriptors are computed over the shapes and applied on a fast configuration of classifiers. Experiments are conducted on three public data sets and the results demonstrate the effectiveness of the method in terms of accuracy and speed.

Due to the large diversity of existing feature descriptors in content-based image retrieval, the image contents can be better represented by the joint use of several descriptors in order to explore their potentially complementary characteristics. This paper presents and discusses a strategy for fusion of the different multidimensional features involved, based on inverted multi-indices and dedicated to similarity search. Image descriptors are quantized separately and efficiently through dimension reduction techniques, before being combined in the inverted multi-indices. To exhibit its effectiveness, the proposal is evaluated on two datasets having different contents and sizes, facing several state-of-the-art approaches of image descriptor fusion. The obtained results reconfirm that the joint use of several descriptions improves similarity search, and show that our fusion proposal outperforms other solutions, while manipulating lower or similar volumes of features.

Digital images play an essential role in analysis tasks with applications in various knowledge domains, such as medicine, meteorology, geology, biology, among others. Such images can be degraded by noise during the process of acquisition, transmission, storage or compression. Although several image denoising methods have been proposed in the literature, noise suppression in images still remains a challenging problem for researchers since the process can cause the removal of relevant image features, such as edges and corners. This papers describes a novel image denoising method based on a genetic algorithm. A population of noisy images is evolved for several epochs applying tailor-made crossover and mutation operators. The population is reinitialized every time a convergence occurs, when only the best individual (image) is kept for the next epoch. Experimental results demonstrate that the proposed method is competitive in comparison with state-of-the-art approaches.

For decades, photographs have been used to document space-time events and they have often served as evidence in courts. Although photographers are able to create composites of analog pictures, this process is very time consuming and requires expert knowledge. Today, however, powerful digital image editing software makes image modifications straightforward. This undermines our trust in photographs and, in particular, questions pictures as evidence for real-world events. In this paper, we analyze one of the most common forms of photographic manipulation, known as image composition or splicing. We propose a forgery detection method that exploits subtle inconsistencies in the color of the illumination of images. Our approach is machine-learning-based and requires minimal user interaction. The technique is applicable to images containing two or more people and requires no expert interaction for the tampering decision. To achieve this, we incorporate information from physics- and statistical-based illuminant estimators on image regions of similar material. From these illuminant estimates, we extract texture- and edge-based features which are then provided to a machine-learning approach for automatic decision-making. The classification performance using an SVM meta-fusion classifier is promising. It yields detection rates of 86% on a new benchmark dataset consisting of 200 images, and 83% on 50 images that were collected from the Internet.

The use of scalar data has arisen in many image applications which obtain data from simulated experiments, range scanners and photogrammetry. Their different nature of acquisition influences the type of processing and the analysis that may be undertaken in tasks such as registration, retrieval and recognition of structures or objects. This paper presents a new method for detecting and describing scale invariant descriptors over dense scalar data sets. The detection of interest points for each image is accomplished using an approach based on the Morse theory and descriptors are computed exploring the geometry of the data. Experiments are performed to demonstrate the effectiveness of the proposed method.

Abstract Dimensionality reduction is employed for visual data analysis as a way to obtaining reduced spaces for high dimensional data or to mapping data directly into 2D or 3D spaces. Although techniques have evolved to improve data segregation on reduced or visual spaces, they have limited capabilities for adjusting the results according to user's knowledge. In this paper, we propose a novel approach to handling both dimensionality reduction and visualization of high dimensional data, taking into account user's input. It employs Partial Least Squares (PLS), a statistical tool to perform retrieval of latent spaces focusing on the discriminability of the data. The method employs a training set for building a highly precise model that can then be applied to a much larger data set very effectively. The reduced data set can be exhibited using various existing visualization techniques. The training data is important to code user's knowledge into the loop. However, this work also devises a strategy for calculating PLS reduced spaces when no training data is available. The approach produces increasingly precise visual mappings as the user feeds back his or her knowledge and is capable of working with small and unbalanced training sets.

Spoofing identities using photographs is one of the most common techniques to attack 2-D face recognition systems. There seems to exist no comparative studies of different techniques using the same protocols and data. The motivation behind this competition is to compare the performance of different state-of-the-art algorithms on the same database using a unique evaluation method. Six different teams from universities around the world have participated in the contest. Use of one or multiple techniques from motion, texture analysis and liveness detection appears to be the common trend in this competition. Most of the algorithms are able to clearly separate spoof attempts from real accesses. The results suggest the investigation of more complex attacks.

Personal identity verification based on biometrics has received increasing attention since it allows reliable authentication through intrinsic characteristics, such as face, voice, iris, fingerprint, and gait. Particularly, face recognition techniques have been used in a number of applications, such as security surveillance, access control, crime solving, law enforcement, among others. To strengthen the results of verification, biometric systems must be robust against spoofing attempts with photographs or videos, which are two common ways of bypassing a face recognition system. In this paper, we describe an anti-spoofing solution based on a set of low-level feature descriptors capable of distinguishing between 'live' and 'spoof images and videos. The proposed method explores both spatial and temporal information to learn distinctive characteristics between the two classes. Experiments conducted to validate our solution with datasets containing images and videos show results comparable to state-of-the-art approaches.

Problems such as image classification, object detection and recognition rely on low-level feature descriptors to represent visual information. Several feature extraction methods have been proposed, including the Histograms of Oriented Gradients (HOG), which captures edge information by analyzing the distribution of intensity gradients and their directions. In addition to directions, the analysis of edge at different scales provides valuable information. Shearlet transforms provide a general framework for analyzing and representing data with anisotropic information at multiple scales. As a consequence, signal singularities, such as edges, can be precisely detected and located in images. Based on the idea of employing histograms to estimate the distribution of edge orientations and on the accurate multi-scale analysis provided by shearlet transforms, we propose a feature descriptor called Histograms of Shearlet Coefficients (HSC). Experimental results comparing HOG with HSC show that HSC provides significantly better results for the problems of texture classification and face identification.

An alternative form to multidimensional projections for the visual analysis of data represented in multidimensional spaces is the deployment of similarity trees, such as Neighbor Joining trees. They organize data objects on the visual plane emphasizing their levels of similarity with high capability of detecting and separating groups and subgroups of objects. Besides this similarity-based hierarchical data organization, some of their advantages include the ability to decrease point clutter; high precision; and a consistent view of the data set during focusing, offering a very intuitive way to view the general structure of the data set as well as to drill down to groups and subgroups of interest. Disadvantages of similarity trees based on neighbor joining strategies include their computational cost and the presence of virtual nodes that utilize too much of the visual space. This paper presents a highly improved version of the similarity tree technique. The improvements in the technique are given by two procedures. The first is a strategy that replaces virtual nodes by promoting real leaf nodes to their place, saving large portions of space in the display and maintaining the expressiveness and precision of the technique. The second improvement is an implementation that significantly accelerates the algorithm, impacting its use for larger data sets. We also illustrate the applicability of the technique in visual data mining, showing its advantages to support visual classification of data sets, with special attention to the case of image classification. We demonstrate the capabilities of the tree for analysis and iterative manipulation and employ those capabilities to support evolving to a satisfactory data organization and classification.

Image segmentation is a primary step in many computer vision tasks. Although many segmentation methods have been proposed in the last decades, there is no generic method that can be applied in a great variety of images. This work presents a new image segmentation method using texture features extracted by wavelet transforms combined with spatial dependence modeled by a Markov random field (MRF). The method initially produces a coarse segmentation, which is refined through a relaxation method based on a new energy function. A set of textured images is used to demonstrate the effectiveness of the proposed method.

Abstract Triangle mesh simplification is of great interest in a variety of knowledge domains, since it allows manipulation and visualization of large models, and it is the starting point for the design of many multiresolution representations. A crucial point in the structure of a simplification method is the definition of an appropriate metric for guiding the decimation process, with the purpose of generating low error approximations at different levels of resolution. This paper proposes two new alternative metrics for mesh simplification, with the aim of producing high‐quality results with reduced execution time and memory usage, and being simple to implement. A set of different established metrics is also described and a comparative evaluation of these metrics against the two new metrics is performed. A single implementation is used in the experiments, in order to enable the evaluation of these metrics independently from other simplification aspects. Results obtained from the simplification of a number of models, using the different metrics, are compared.

This paper presents a method for classification of textures based on features obtained from co-occurrence matrices and Markov random fields. Two steps are performed to classify the images. Initially, the method recognizes the homogeneous regions (object interior) in the image. Regions consisting of dissimilar elements (transition between objects) are then properly identified and classified. Experimental results demonstrate the robustness of the method in terms of variation in region size and number of parameters.

The approximation of surfaces to scattered data is an important problem encountered in a variety of scientific applications, such as reverse engineering, computer vision, computer graphics, and terrain modeling. This paper describes an automatic method for constructing smooth surfaces defined as a network of curved triangular patches. The method starts with a coarse mesh approximating the surface through triangular elements covering the boundary of the domain, then iteratively adds new points from the data set until a specified error tolerance is achieved. The resulting surface over the triangular mesh is represented by piecewise polynomial patches possessing C/sup 1/ continuity. The method has been implemented and tested on a number of real data sets.

A method is described for the extraction of morphological information from images approximated by triangular meshes. Topographic features such as peaks, pits, ridges, valleys, and planar regions are considered the basic descriptive surface elements and are defined in terms of the local organization of the triangles in the mesh. The approach is suitable for image analysis tasks, simplifying object recognition and scene interpretation. Several images have been used to demonstrate the performance of the proposed method.