Partial retrieval is a long-standing problem in the 3D Object Retrieval community. Its main difficulties arise from how to define 3D local descriptors in a way that makes them effective for partial retrieval and robust to common real-world issues, such as occlusion, noise, or clutter, when dealing with 3D data. This SHREC track is based on the newly proposed ShapeBench benchmark to evaluate the matching performance of local descriptors. We propose an experiment consisting of three increasing levels of difficulty, where we combine different filters to simulate real-world issues related to the partial retrieval task. Our main findings show that classic 3D local descriptors like Spin Image are robust to several of the tested filters (and their combinations), but more recent learned local descriptors like GeDI can be competitive for some specific filters. Finally, no 3D local descriptor was able to successfully handle the hardest level of difficulty. • We evaluate the robustness of local 3D descriptors for partial shape retrieval using a novel benchmark—ShapeBench—under progressively challenging conditions simulating real-world degradations (e.g., clutter, occlusion, noise, remeshing). • Our analysis shows that classic hand-crafted descriptors like Spin Image consistently outperform more recent learned descriptors under high levels of occlusion and noise. • No existing local 3D descriptor was found to be reliably effective under the most challenging scenarios combining multiple perturbations, highlighting an open problem in robust partial 3D retrieval.

We extend the concept of worst-case optimal equijoins in graph databases to the case where some nodes are required to be within the k-nearest neighbors (kNN) of others under some similarity function. We model the problem by superimposing the database graph with the kNN graph and show that a variant of Leapfrog TrieJoin (LTJ) implemented over a compact data structure called the Ring can be seamlessly extended to integrate similarity clauses with the equijoins in the LTJ query process, retaining worst-case optimality in many relevant cases. Our experiments on a benchmark that combines Wikidata and IMGpedia show that our enhanced LTJ algorithm outperforms by a considerable margin a baseline that first applies classic LTJ and then completes the query by applying the similarity predicates. The difference is more pronounced on queries where the similarity clauses are more densely connected to the query, becoming of an order of magnitude in some cases.

Among all Internet Quality of Service (QoS) indicators, Round-trip time (RTT), jitter and packet loss have been thoroughly studied due to their great impact on the overall network's performance and the Quality of Experience (QoE) perceived by the users. Considering that, we managed to generate a real-world dataset with a comprehensive contextualization of these important quality indicators by passively monitoring the network in user-space. To generate this dataset, we first developed a novel Periodic Passive Ping (PePa Ping) methodology for Android devices. Contrary to other works, PePa Ping periodically obtains RTT, jitter, and number of lost packets of all TCP connections. This passive approach relies on the implementation of a local VPN server residing inside the client device to manage all Internet traffic and obtain QoS information of the connections established. The collected QoS indicators are provided directly by the Linux kernel, and therefore, they are exceptionally close to real QoS values experienced by users' devices. Additionally, the PePa Ping application continuously measured other indicators related to each individual network flow, the state of the device, and the state of the Internet connection (either WiFi or Mobile). With all the collected information, each network flow can be precisely linked to a set of environmental data that provides a comprehensive contextualization of each individual connection.

Video captioning is the task of predicting a semantic and syntactically correct sequence of words given some context video. The most successful methods for video captioning have a strong dependency on the effectiveness of semantic representations learned from visual models, but often produce syntactically incorrect sentences which harms their performance on standard datasets. In this paper, we address this limitation by considering syntactic representation learning as an essential component of video captioning. We construct a visual-syntactic embedding by mapping into a common vector space a visual representation, that depends only on the video, with a syntactic representation that depends only on Part-of-Speech (POS) tagging structures of the video description. We integrate this joint representation into an encoder-decoder architecture that we call Visual-Semantic-Syntactic Aligned Network (SemSynAN), which guides the decoder (text generation stage) by aligning temporal compositions of visual, semantic, and syntactic representations. We tested our proposed architecture obtaining state-of-the-art results on two widely used video captioning datasets: the Microsoft Video Description (MSVD) dataset and the Microsoft Research Video-to-Text (MSR-VTT) dataset.

Abstract In digital archaeology, a large research area is concerned with the computer‐aided analysis of 3D captured ancient pottery objects. A key aspect thereby is the analysis of motifs and patterns that were painted on these objects' surfaces. In particular, the automatic identification and segmentation of repetitive patterns is an important task serving different applications such as documentation, analysis and retrieval. Such patterns typically contain distinctive geometric features and often appear in repetitive ornaments or friezes, thus exhibiting a significant amount of symmetry and structure. At the same time, they can occur at varying sizes, orientations and irregular placements, posing a particular challenge for the detection of similarities. A key prerequisite to develop and evaluate new detection approaches for such repetitive patterns is the availability of an expressive dataset of 3D models, defining ground truth sets of similar patterns occurring on their surfaces. Unfortunately, such a dataset has not been available so far for this particular problem. We present an annotated dataset of 82 different 3D models of painted ancient Peruvian vessels, exhibiting different levels of repetitiveness in their surface patterns. To serve the evaluation of detection techniques of similar patterns, our dataset was labeled by archaeologists who identified clearly definable pattern classes. Those given, we manually annotated their respective occurrences on the mesh surfaces. Along with the data, we introduce an evaluation benchmark that can rank different recognition techniques for repetitive patterns based on the mean average precision of correctly segmented 3D mesh faces. An evaluation of different incremental sampling‐based detection approaches, as well as a domain specific technique, demonstrates the applicability of our benchmark. With this benchmark we especially want to address the geometry processing community, and expect it will induce novel approaches for pattern analysis based on geometric reasoning like 2D shape and symmetry analysis. This can enable novel research approaches in the Digital Humanities and related fields, based on digitized 3D Cultural Heritage artifacts. Alongside the source code for our evaluation scripts we provide our annotation tools for the public to extend the benchmark and further increase its variety.

The Domain Name System (DNS) is a critical component of Internet infrastructure, as almost every activity on the Internet starts with a DNS query. Given its importance, there is increasing concern over its vulnerability to attacks and failures, as they can negatively affect all Internet-based resources. Thus, detecting these events is crucial to preserve the correct functioning of all DNS components, such as high-volume name servers for top-level domains (TLD). This article presents a near real-time Anomaly Detection Based on Prediction (AD-BoP) method, providing a useful and easily explainable methodology to effectively detect DNS anomalies. AD-BoP is based on the prediction of expected DNS traffic statistics, and could be especially helpful for TLD registry operators to preserve their services' reliability. After an exhaustive analysis, AD-BoP is shown to improve the current state-of-the-art for anomaly detection in authoritative TLD name servers.

As an essential high-level task of video understanding topic, automatically describing a video with natural language has recently gained attention as a fundamental challenge in computer vision. Previous models for video captioning have several limitations, such as the existence of gaps in current semantic representations and the inexpressibility of the generated captions. To deal with these limitations, in this paper, we present a new architecture that we call Attentive Visual Semantic Specialized Network (AVSSN), which is an encoder-decoder model based on our Adaptive Attention Gate and Specialized LSTM layers. This architecture can selectively decide when to use visual or semantic information into the text generation process. The adaptive gate makes the decoder to automatically select the relevant information for providing a better temporal state representation than the existing decoders. Besides, the model is capable of learning to improve the expressiveness of generated captions attending to their length, using a sentence-length-related loss function. We evaluate the effectiveness of the proposed approach on the Microsoft Video Description (MSVD) and the Microsoft Research Video-to-Text (MSR-VTT) datasets, achieving state-of-the-art performance with several popular evaluation metrics: BLEU-4, METEOR, CIDEr, and ROUGEL.

This work presents and studies the efficiency problem of mapping GPU threads onto simplex domains. A non-linear map$\lambda (\omega)$is formulated based on a block-space enumeration principle that reduces the number of thread-blocks by a factor of approximately$2\times$and$6\times$for 2-simplex and 3-simplex domains, respectively, when compared to the standard approach. Performance results show that$\lambda (\omega)$is competitive and even the fastest map when ran in recent GPU architectures such as the Tesla V100, where it reaches up to$1.5\times$of speedup in 2-simplex tests. In 3-simplex tests, it reaches up to$2.3\times$of speedup for small workloads and up to$1.25\times$for larger ones. The results obtained make$\lambda (\omega)$a useful GPU optimization technique with applications on parallel problems that define all-pairs, all-triplets or nearest neighbors interactions in a 2-simplex or 3-simplex domain.

The study of data-parallel domain re-organization and thread-mapping techniques are relevant topics as they can increase the efficiency of GPU computations on spatial discrete domains with non-box-shaped geometry. In this work we study the potential benefits of applying a succinct data re-organization of a tetrahedral data-parallel domain of size O(n3) combined with an efficient block-space GPU map of the form g (λ) : N → N3. Results from the analysis suggest that in theory the combination of these two optimizations produce significant performance improvement as block-based data reorganization allows a coalesced one-to-one correspondence at local thread-space while g(λ) produces an efficient block-space spatial correspondence between groups of data and groups of threads, reducing the number of unnecessary threads from O(n3) to O(n2ρ3) with ρ ∊ O(1). From the analysis, we obtained that a block based succinct data re-organization can provide up to 2× improved performance over a linear data organization while the map can be up to 6× more efficient than a bounding box approach. The results from this work can serve as a useful guide for a more efficient GPU computation on tetrahedral domains found in spin lattice, finite element and special n-body problems, among others.

This paper presents a hierarchical method for finding correspondences in non-rigid shapes. We propose a new representation for 3D meshes: the decomposition tree. This structure characterizes the recursive decomposition process of a mesh into regions of interest and key points. The internal nodes contain regions of interest (which may be recursively decomposed) and the leaf nodes contain the key points to be matched. We also propose a hierarchical matching algorithm that performs in a level-wise manner. The matching process is guided by the similarity between regions in high levels of the tree, until reaching the key points stored in the leaves. This allows us to reduce the search space of correspondences, making also the matching process efficient. We evaluate the effectiveness of our approach using the SHREC'2010 robust correspondence benchmark. In addition, we show that our results outperform the state of the art.

In this paper, we present a new approach for generic 3D shape retrieval based on a mesh partitioning scheme. Our method combines a mesh global description and mesh partition descriptions to represent a 3D shape. The partitioning is useful because it helps us to extract additional information in a more local sense. Thus, part descriptions can mitigate the semantic gap imposed by global description methods. We propose to find spatial agglomerations of local features to generate mesh partitions. Hence, the definition of a distance function is stated as an optimization problem to find the best match between two shape representations. We show that mesh partitions are representative and therefore it helps to improve the effectiveness in retrieval tasks. We present exhaustive experimentation using the SHREC'09 Generic Shape Retrieval Benchmark.

3D Object Retrieval is an important field of research with many application possibilities. One of the main goals in this research is the development of discriminative methods for similarity search. The descriptor-based approach to date has seen a lot of research attention, with many different extraction algorithms proposed. In previous work, we have introduced a simple but effective scheme for 3D model retrieval based on a spatially fixed combination of 3D object fragment descriptors. In this work, we propose a novel flexible combination scheme based on finding the best matching fragment descriptors to use in the combination. By an exhaustive experimental evaluation on established benchmark data we show the capability of the new combination scheme to provide improved retrieval effectiveness. The method is proposed as a versatile and inexpensive method to enhance the effectiveness of a given global 3D descriptor approach.

The caching of accessed disk pages has been successfully used for decades in database technology, resulting in effective amortization of I/O operations needed within a stream of query or update requests. However, in modern complex databases, like multimedia databases, the I/O cost becomes a minor performance factor. In particular, metric access methods (MAMs), used for similarity search in complex unstructured data, have been designed to minimize rather the number of distance computations than I/O cost (when indexing or querying). Inspired by I/O caching in traditional databases, in this paper we introduce the idea of distance caching for usage with MAMs—a novel approach to streamline similarity search. As a result, we present the D-cache, a main-memory data structure which can be easily implemented into any MAM, in order to spare the distance computations spent by queries/updates. In particular, we have modified two state-of-the-art MAMs to make use of D-cache—the M-tree and Pivot tables. Moreover, we present the D-file, an index-free MAM based on simple sequential search augmented by D-cache. The experimental evaluation shows that performance gain achieved due to D-cache is significant for all the MAMs, especially for the D-file.

This tutorial surveys domains employing non-metric functions for effective similarity search, and methods for efficient non-metric similarity search in very large collections.

Content-Based Video Copy Detection (CBVCD) consists of detecting and retrieving videos that are copies of known original videos. CBVCD systems rely on two different tasks: Feature Extraction task, that calculates many representative descriptors for a video sequence, and Similarity Search task, that is the algorithm for finding videos in an indexed collection that match a query video. This paper describes P-VCD, which is a novel approach for CBVCD based on global de scriptors, weighted combinations of distances, a pivot-based index structure, an approximate similarity search, and a voting algorithm for copy localization. P-VCD was tested at the TRECVID 2010 evaluation, where it was the best positioned CBVCD system for Balanced and No False Alarms profiles considering visual-only runs (and above the median considering all runs). P-VCD shows that by using approximate similarity searches one can obtain good effectiveness, and that global descriptors can achieve competitive results with TRECVID transformations.

Since 3D models are becoming more popular, the need for effective methods capable of retrieving 3D models are becoming crucial. Current methods require an example 3D model as query. However, in many cases, such a query is not easy to get. An alternative is using a hand-draw sketch as query. We present a structure-based local approach (STELA) for retrieving 3D models using a rough sketch as query. It consists of four steps: get an abstract image, detect keyshapes, compute a local descriptor, and match local descriptors. We represent a 3D model by means of suggestive contours. Our proposal includes an additional step aiming at reducing the number of models that will be compared by our local approach. The proposed method is invariant to position, scale, and rotation changes as well. We evaluate our method using the first-tier precision and compare it with a current global approach (HELO). Our results show an increasing in precision for many classes of 3D models.

Content-Based Multimedia Information Retrieval retrieves multimedia documents based on their content (colors, edges, textures, etc.). The content of a whole multimedia document is represented by a global descriptor. The similarity of two multimedia documents can be defined as the distance between their descriptors. A multi-metric function that combines distances from many descriptors usually outperforms the effectiveness of any single descriptor. In this case, a different weight is assigned to each descriptor representing its relative importance in the combination. Usually, these sets of weights are fixed manually or by performing many effectiveness evaluations. In this work, we present three novel techniques for weighting multi-metrics: á-normalization, which is a generalization of the normalization by maximum distance that uses the histogram of distances, MID-weighting which selects weights that maximize intrinsic dimensionality, and MID-á-weighting that combines the two previous techniques. These techniques enable the selection of a set of weights with satisfactory effectiveness without performing any effectiveness evaluation. Thus, they are suitable when a ground truth does not exist or when it is expensive to perform an evaluation. We tested their effectiveness on a content-based copy detection corpus, and we analyzed the behavior of effectiveness and efficiency in a multi-metric space. We conclude that MID-á-weighting outperforms the widely used maximum distance normalization, and that it can be used as an automatic weight selection for further manual adjustment.

We explore the application of a graph representation to model similarity relationships that exist among images found on the Web. The resulting similarity-induced graph allows us to model in a unified way different types of content-based similarities, as well as semantic relationships. Content-based similarities include different image descriptors, and semantic similarities can include relevance user feedback from search engines. The goal of our representation is to provide an experimental framework for combining apparently unrelated metrics into a unique graph structure, which allows us to enhance the results of Web image retrieval. We evaluate our approach by re-ranking Web image search results.

The research on multimedia databases involves different areas in Computer Science, such as computer graphics, databases, and information retrieval. There are many practical applications that benefit from this research, e.g., molecular biology, medicine, CAD/CAM, and geography. An important characteristic of these applications is the variety of data that should be supported, e.g., text, images (both still and moving), and audio. This implies that the development of a multimedia information system is considerably more complex than a traditional information system. An important research issue in the field of multimedia databases is the content-based retrieval of similar objects. Given a multimedia query object, the search for an exact match in a database is not meaningful in most applications, because the probability that two multimedia objects are identical is negligible (unless they are digital copies from the same source). For this reason, the development of efficient and effective similarity search techniques has become an important topic in the multimedia database research community. The goal of this advanced technology seminar is to provide an overview of the similarity search problem and to present the state-of-art techniques for performing efficient and effective similarity queries in multimedia databases. The seminar begins with an introduction and a motivation of multimedia databases. The two main approaches for describing multimedia objects (as elements in a metric space or in a vector space) are introduced, as well as a description of the ”Multimedia Content Description Interface” (MPEG)-7 standard. The efficiency issue is addressed for both metric and vector space approaches, describing the data structures and algorithms used to answer similarity queries. For the effectiveness issue, the seminar introduces some widely used retrieval performance measures. Several examples of techniques for particular multimedia applications (text, image, CAD, 3D objects, audio and video) are presented. The seminar outline is as follows:

Similarity search in 3D object databases is becoming an important problem in multimedia retrieval, with many practical applications. We investigate methods for improving the effectiveness in a retrieval system that implements multiple feature extraction algorithms to choose from. Our techniques are based on the entropy impurity measure, widely used in the context of decision trees. We propose a method for the a priori estimation of individual feature vector performance, given a query. We then define two approaches that use this estimator to improve the retrieval effectiveness. Our experimental results show that significant improvements are achievable using these methods.

Multimedia objects are often described by high-dimensional feature vectors which can be used for retrieval and clustering tasks. We have built an interactive retrieval system for 3D model databases that implements a variety of different feature transforms. Recently, we have enhanced the functionality of our system by integrating a SOM-based visualization module. In this poster demo, we show how 2D maps can be used to improve the effectiveness of retrieval, clustering, and over-viewing tasks in a 3D multimedia system.