Unsupervised domain adaptation (UDA) in videos is a challenging task that remains not well explored compared to image-based UDA techniques. Although vision transformers (ViT) achieve state-of-the-art performance in many computer vision tasks, their use in video UDA has been little explored. Our key idea is to use transformer layers as a feature encoder and incorporate spatial and temporal transferability relationships into the attention mechanism. A Transferable-guided Attention (TransferAttn) framework is then developed to exploit the capacity of the transformer to adapt cross-domain knowledge across different backbones. To improve the transferability of ViT, we introduce a novel and effective module, named Domain Transferable-guided Attention Block (DTAB). DTAB compels ViT to focus on the spatio-temporal transferability relationship among video frames by changing the self-attention mechanism to a transferability attention mechanism. Extensive experiments were conducted on UCF-HMDB, Kinetics-Gameplay, and Kinetics-NEC Drone datasets, with different backbones, like ResNet101, I3D, and STAM, to verify the effectiveness of TransferAttn compared with state-of-the-art approaches. Also, we demonstrate that DTAB yields performance gains when applied to other state-of-the-art transformer-based UDA methods from both video and image domains. Our code is available at https://github.com/Andre-Sacilotti/transferattn-project-code.

Deep learning (DL) technologies can transform agriculture by improving crop health monitoring and management, thus improving food safety. In this letter, we explore the potential of edge computing (EC) for real-time classification of leaf diseases using thermal imaging. We present a thermal image dataset for plant disease classification and evaluate DL models, including InceptionV3, MobileNetV1, MobileNetV2, and VGG-16, on resource-constrained devices like the Raspberry Pi 4B. Using pruning and quantization-aware training, these models achieve inference times up to$1.48\times $faster on Edge TPU Max for VGG16, and up to$2.13\times $faster with precision reduction on Intel NCS2 for MobileNetV1, compared with high-end GPUs like RTX 3090, while maintaining state-of-the-art accuracy.

Patients with dysarthria face challenges in verbal communication, which affects their interaction with speech-activated devices. Intelligent systems that can interpret dysarthric speech could significantly enhance their quality of life. Neural Networks (NN) and Convolutional Neural Networks (CNN) have been used for sparse word classification in dysarthric speech, achieving an average accuracy of 64.1%. Spatial Convolutional Neural Networks (SCNN) and Multi-Head Attention Transformers (MHAT) have improved this accuracy by 20%. However, these methods have been tested on limited databases and yield specific results, making their application in more natural speech environments challenging. To address the lack of dysarthric speech data, some researchers have used speech synthesis techniques, but they require extensive training and careful database structuring. To alleviate this problem, we explore data augmentation methods based on the spectral characteristics of dysarthric speech, which focus on transforming spectrographic images extracted from available audio files. Two such methods were implemented and achieved results similar to complex state-of-the-art solutions.

In precision agriculture, detecting productive crop fields is an essential practice that allows the farmer to evaluate operating performance separately and compare different seed varieties, pesticides, and fertilizers. However, manually identifying productive fields is often time-consuming, costly, and subjective. Previous studies explore different methods to detect crop fields using advanced machine learning algorithms to support the specialists’ decisions, but they often lack good quality labeled data. In this context, we propose a high-quality dataset generated by machine operation combined with Sentinel-2 images tracked over time. As far as we know, it is the first one to overcome the lack of labeled samples by using this technique. In sequence, we apply a semi-supervised classification of unlabeled data and state-of-the-art supervised and self-supervised deep learning methods to detect productive crop fields automatically. Finally, the results demonstrate high accuracy in Positive Unlabeled learning, which perfectly fits the problem where we have high confidence in the positive samples. Best performances have been found in Triplet Loss Siamese given the existence of an accurate dataset and Contrastive Learning considering situations where we do not have a comprehensive labeled dataset available.

Deep learning architectures have achieved promising results in different areas (e.g., medicine, agriculture, and security). However, using those powerful techniques in many real applications becomes challenging due to the large labeled collections required during training. Several works have pursued solutions to overcome it by proposing strategies that can learn more for less, e.g., weakly and semi-supervised learning approaches. As these approaches do not usually address memorization and sensitivity to adversarial examples, this paper presents three deep metric learning approaches combined with Mixup for incomplete-supervision scenarios. We show that some state-of-the-art approaches in metric learning might not work well in such scenarios. Moreover, the proposed approaches outperform most of them in different datasets.

Most image data available are often stored in a compressed format, from which JPEG is the most widespread. To feed this data on a convolutional neural network (CNN), a preliminary decoding process is required to obtain RGB pixels, demanding a high computational load and memory usage. For this reason, the design of CNNs for processing JPEG compressed data has gained attention in recent years. In most existing works, typical CNN architectures are adapted to facilitate the learning with the DCT coefficients rather than RGB pixels. Although they are effective, their architectural changes either raise the computational costs or neglect relevant information from DCT inputs. In this paper, we examine different ways of speeding up CNNs designed for DCT inputs, exploiting learning strategies to reduce the computational complexity by taking full advantage of DCT inputs. Our experiments were conducted on the ImageNet dataset. Results show that learning how to combine all DCT inputs in a data-driven fashion is better than discarding them by hand, and its combination with a reduction of layers has proven to be effective for reducing the computational costs while retaining accuracy.

Over the past decade, convolutional neural networks (CNNs) have achieved state-of-the-art performance in many computer vision tasks. They can learn robust representations of image data by processing RGB pixels. Since image data are often stored in a compressed format, from which JPEG is the most widespread, a preliminary decoding process is demanded. Recently, the design of CNNs for processing JPEG compressed data has gained attention from the research community. They process DCT coefficients instead of RGB pixels, saving computation for decoding JPEG images, however, at the cost of increasing the computational complexity of the network. In this paper, we examine how spatial resolution and JPEG quality impacts on the performance of a state-of-the-art CNN designed to operate directly on the JPEG compressed domain. To alleviate its computational complexity, we propose a Frequency Band Selection (FBS) technique to select the most relevant DCT coefficients before feeding them to the network. Experiments were conducted on a subset of the ImageNet dataset considering both fine- and coarse-grained image classification tasks. Results show that such networks are resilient to JPEG quality but are susceptible to spatial resolution. Also, our FBS can reduce the computational complexity of the network while retaining a similar accuracy.

Accurately ranking images and multimedia objects are of paramount relevance in many retrieval and learning tasks. Manifold learning methods have been investigated for ranking mainly due to their capacity of taking into account the intrinsic global manifold structure. In this paper, a novel manifold ranking algorithm is proposed based on the hypergraphs for unsupervised multimedia retrieval tasks. Different from traditional graph-based approaches, which represent only pairwise relationships, hypergraphs are capable of modeling similarity relationships among a set of objects. The proposed approach uses the hyperedges for constructing a contextual representation of data samples and exploits the encoded information for deriving a more effective similarity function. An extensive experimental evaluation was conducted on nine public datasets including diverse retrieval scenarios and multimedia content. Experimental results demonstrate that high effectiveness gains can be obtained in comparison with the state-of-the-art methods.

The increasing amount of multimedia data collections available today evinces the pressing need for methods capable of indexing and retrieving this content. Despite the continuous advances in multimedia features and representation models, to establish an effective measure for comparing different multimedia objects still remains a challenging task. While supervised and semi-supervised techniques made relevant advances on similarity learning tasks, scenarios where labeled data are non-existent require different strategies. In such situations, unsupervised learning has been established as a promising solution, capable of considering the contextual information and the dataset structure for computing new similarity/dissimilarity measures. This article extends a recent unsupervised learning algorithm that uses an iterative re-ranking strategy to take advantage of different k -Nearest Neighbors (kNN) sets and rank correlation measures. Two novel approaches are proposed for computing the kNN sets and their corresponding top- k lists. The proposed approaches were validated in conjunction with various rank correlation measures, yielding superior effectiveness results in comparison with previous works. In addition, we also evaluate the ability of the method in considering different multimedia objects, conducting an extensive experimental evaluation on various image and video datasets.

We present a novel approach for phase denoising in Interferometric Synthetic Aperture Radar (InSAR) images, named as Block-Matching InSAR (BMInSAR). It uses k-means clustering to solve the block matching similarity search problem, thus simplifying preprocessing steps and filtering several reference-blocks at once. Also, we propose a novel methodology based on ground-truth GPS measurements to assess the filtering quality of Digital Elevation Models (DEMs) derived from a pair of Very High-Resolution (VHR) SAR complex images. Our dataset was obtained by X-Band airborne sensor OrbiSAR-2 from BRADAR. BMInSAR significantly outperforms the state-of-the-art filtering methods in both accuracy and execution time. After filtering with BMInSAR, we achieved an accuracy of 21cm in the resulting DEM of a homogeneous lawn area, which is quite similar to that obtained by LiDAR technology.

Remote sensing has been widely applied for environmental monitoring by means of change detection techniques, commonly for identifying deforestation signs which is the gateway for illegal activities such as uncontrolled urban growth and grazing pasture. Monthly acquired X-Band images from airborne Synthetic Aperture Radar (SAR) provided multi-temporal scenes employed in this work resulting in environmental incident reports forwarded to the responsible authorities. The present work proposes the use of both, Superpixel segmentation by Simple Linear Iterative Clustering (SLIC) and change detection by Object Correlation Images (OCI) not yet applied to multi-temporal X-Band high resolution SAR images, and the application of a simple Multilayer Perceptron (MLP) supervised learning technique for detecting and classifying the changes into relevant activities. Experiments have been performed using acquired SAR imagery from BRADAR airborne sensor OrbiSAR-2 under Brazilian Atlantic Forest which revealed possible deforestation activities comparing achieved results with those obtained with experts.

PhenoVis is framework for the visual phenological analysis of forest ecosystems. It contains the chronological percentage maps (CPM), a novel representation that is capable of discovering additional patterns by encoding percentage distributions of the data. Two types of masks are used in PhenoVis: a community mask , which considers all plant species in the image; and a species mask , associated with a given plant species. Among the several images taken at different times of the day, the image taken at noon is preferred for the analysis because it minimizes shadow effects. Therefore, only one image per day is used. The analysis considers the chromatic co- efficients associated with each pixel in the image. In PhenoVis we associate different colors with each bucket of the percentage histogram. The histogram granularity defines the size of a given bucket of the percentage distribution. The number of buckets is given by the number of colors available, and the range of the distribution is given by the IOI. The percentage map of a single input image consists of a normalized stacked bar chart. The chronological percentage map consists of a sequence of percentage maps stacked in chronological order, from top to bottom (portrait) or left to right (landscape).

Various unsupervised learning methods have been proposed with significant improvements in the effectiveness of image search systems. However, despite the relevant effectiveness gains, these approaches commonly require high computation efforts, not addressing properly efficiency and scalability requirements. In this paper, we present a novel unsupervised learning approach for improving the effectiveness of image retrieval tasks. The proposed method is also scalable and efficient as it exploits parallel and heterogeneous computing on CPU and GPU devices. Extensive experiments were conducted considering five different public image collections and several descriptors. This rigorous experimental protocol evaluates the effectiveness, efficiency, and scalability of the proposed approach, and compares it with previous methods. Experimental results demonstrate that high effectiveness gains (up to +29%) can be obtained requiring small run times.

Plant phenology has been exploited as an important research venue for assessing the impact of climate changes. One common approach for monitoring vegetation relies on the use of digital cameras. The employment of imaging techniques for phenological observation allows the extraction and analysis of visual characteristics based on color and texture information with the objective of determining plant life cycle changes, such as the beginning of the leaf flushing or the senescence period. This paper presents a novel approach for detecting phenological changes by analyzing image temporal series. Our method is based on the use of visual rhythm analysis and the adoption of a dissimilarity measure to detect visual changes in the time line. Experiments were conducted on a three-year data set composed of 3,538 vegetation images and 21 samples of 6 different species of interest. Results demonstrate that the proposed change detection approach is able to effectively identify phenological events.

Phenology is the study of periodic natural phenomena and their relationship to climate. Usually, phenology studies consider the identification of patterns on temporal data. In those studies, several phenological change patterns are often encoded in time series for analysis and knowledge extraction. In this paper, we evaluate the effectiveness of several time series similarity functions in the task of classifying time series related to phonological phenomena characterized by near-surface vegetation indices extracted from images. In addition, we performed a correlation analysis to identify potential candidates for combination.

Plant phenology studies recurrent plant life cycles events and is a key component of climate change research. To increase accuracy of observations, new technologies have been applied for phenological observation, and one of the most successful are digital cameras, used as multi-channel imaging sensors to estimate color changes that are related to phenological events. We monitored leaf-changing patterns of a cerrado-savanna vegetation by taken daily digital images. We extract individual plant color information and correlated with leaf phenological changes. To do so, time series associated with plant species were obtained, raising the need of using appropriate tools for mining patterns of interest. In this paper, we present a novel approach for representing phenological patterns of plant species derived from digital images. The proposed method is based on encoding time series as a visual rhythm, which is characterized by image description algorithms. A comparative analysis of different descriptors is conducted and discussed. Experimental results show that our approach presents high accuracy on identifying plant species.

Plant phenology has gained importance in the context of global change research, stimulating the development of new technologies for phenological observation. In this context, digital cameras have been successfully used as multi-channel imaging sensors, providing measures to estimate changes on phenological events, such as leaf flushing and senescence. We monitored leaf-changing patterns of a cerrado-savanna vegetation by taken daily digital images. For that, we extract leaf color information and correlated with phenological changes. In this way, time series associated with plant species are obtained, raising the need of using appropriate tools for mining patterns of interest. In this paper, we present a novel approach for representing phenological patterns of plant species. The proposed method is based on encoding time series as a visual rhythm, which is characterized by color description algorithms. A comparative analysis of different descriptors is conducted and discussed. Experimental results show that our approach presents high accuracy on identifying plant species.

Remote phenology has motivated the development of new technologies for pattern observation. In this scenario, digital cameras have been used as data source for studies that estimate changes on phenological events. In this paper, we investigate the use of shape descriptors in the task of characterizing time series associated with phenological changes. The main objectives are: i) to determine which color channel is better for extracting shape descriptors and ii) to analyze the impact of the sunshine on the performance of shape descriptors.

Plant phenology has gained importance in the context of global change research, stimulating the development of new technologies for phenological observation. Digital cameras have been successfully used as multi-channel imaging sensors, providing measures of leaf color change information (RGB channels), or leafing phenological changes in plants. We monitored leaf-changing patterns of a cerrado-savanna vegetation by taken daily digital images. We extract RGB channels from digital images and correlated with phenological changes. Our first goals were: (1) to test if the color change information is able to characterize the phenological pattern of a group of species; and (2) to test if individuals from the same functional group may be automatically identified using digital images. In this paper, we present a machine learning approach to detect phenological patterns in the digital images. Our preliminary results indicate that: (1) extreme hours (morning and afternoon) are the best for identifying plant species; and (2) different plant species present a different behavior with respect to the color change information. Based on those results, we suggest that individuals from the same functional group might be identified using digital images, and introduce a new tool to help phenology experts in the species identification and location on-the-ground.

This work is developed in the context of the placing task of the MediaEval 2011 initiative. The objective is to geocode (or geotag) a set of videos, i.e., automatically assign geographical coordinates to them. This paper presents an architecture for multimodal geocoding that exploits both visual and textual descriptions associated with videos. This work also describes our efforts regarding the implementation of this architecture to demonstrate its applicability. Conducted experiments show how our multimodal approach enhances the results compared to relying on a single modality.

This paper presents a novel approach for video representation, called bag-of-scenes. The proposed method is based on dictionaries of scenes, which provide a high-level representation for videos. Scenes are elements with much more semantic information than local features, specially for geotagging videos using visual content. Thus, each component of the representation model has self-contained semantics and, hence, it can be directly related to a specific place of interest. Experiments were conducted in the context of the MediaEval 2011 Placing Task. The reported results show our strategy compared to those from other participants that used only visual content to accomplish this task. Despite our very simple way to generate the visual dictionary, which has taken photos at random, the results show that our approach presents high accuracy relative to the state-of-the art solutions.

Making efficient use of video information requires the development of a video signature and a similarity measure to rapidly identify similar videos in a huge database. Most of existing techniques to address this problem have focused on the uncompressed domain. However, decoding and analyzing of a video sequence are extremely time-consuming tasks. Since video data are usually available in compressed form, it is desirable to directly process video material without decoding. In this paper, we present a novel approach for comparing video sequences that works in the compressed domain. The proposed method is based on recognizing motion patterns extracted from the video stream and their occurrence histogram is proven to be a powerful feature for describing the video content. Experiments on a TRECVID 2010 dataset show that our approach presents high accuracy relative to the state-of-the-art solutions and in a computational time that makes it suitable for large collections.

Similarity search in high-dimensional metric spaces is a key operation in many applications, such as multimedia databases, image retrieval, object recognition, and others. The high dimensionality of the data requires special index structures to facilitate the search. Most of existing indexes are constructed by partitioning the data set using distance-based criteria. However, those methods either produce disjoint partitions, but ignore the distribution properties of the data; or produce non-disjoint groups, which greatly affect the search performance. In this paper, we study the performance of a new index structure, called Ball-and-Plane tree (BP-tree), which overcomes the above disadvantages. BP-tree is constructed by recursively dividing the data set into compact clusters. Distinctive from other techniques, it integrates the advantages of both disjoint and non-disjoint paradigms in order to achieve a structure of tight and low overlapping clusters, yielding significantly improved performance. Results obtained from an extensive experimental evaluation with real-world data sets show that BP-tree consistently outperforms state-of-the-art solutions.

Recent advances in technology have increased the availability of video data, creating a strong requirement for efficient systems to manage those materials. Making efficient use of video information requires that data be accessed in a user-friendly way. This has been the goal of a quickly evolving research area known as video summarization. Most of existing techniques to address the problem of summarizing a video sequence have focused on the uncompressed domain. However, decoding and analyzing of a video sequence are two extremely time-consuming tasks. Since video data are usually available in compressed form, it is desirable to directly process video material without decoding. In this paper, we present a novel approach for video summarization that works in the compressed domain. The proposed method is based on both exploiting visual features extracted from the video stream and on using a simple and fast algorithm to summarize the video content. Results from a rigorous empirical comparison with a subjective evaluation show that our approach produces video summaries with superior quality relative to the state-of-the-art solutions and in a computational time that allows on-the-fly usage.

There are different approaches able to automatically detect e-mail spam messages, and the best-known ones are based on Bayesian decision theory. However, the most of these approaches have the same difficulty: the high dimensionality of the feature space. Many term selection methods have been proposed in the literature. Nevertheless, it is still unclear how the performance of naive Bayes anti-spam filters depends on the methods applied for reducing the dimensionality of the feature space. In this paper, we compare the performance of most popular methods used as term selection techniques, such as document frequency, information gain, mutual information, X2statistic, and odds ratio used for reducing the dimensionality of the term space with four well-known different versions of naive Bayes spam filter.