5583

5642

Data science projects require the coordination of diverse skills across interdisciplinary teams. As organizations increasingly adopt automated tools for planning and staffing, decision support systems (DSS) are becoming more relevant. However, most existing DSS focus on technical workflows and neglect human competencies. This results in recommendations that may not align with actual team capabilities. This mismatch primarily impacts the success of the project. To address this issue, we propose CARE, a Competence-Aware Requirement Engineering framework that grounds the DSS design on validated user needs. Through expert interviews, we extract competence-related user stories and identify additional system requirements related to decision support functions, interfaces, and competency modeling. These results are validated through a structured survey with 195 professionals involved in data science projects. The results highlight the need for DSS features that support role-competence alignment, identify competency gaps, and provide targeted training. The respondents also prioritized system qualities such as adaptability, explainability, and integration into existing workflows. Based on this multilevel validation, we present a set of empirically grounded requirements for competence-based DSS. The CARE framework provides practical guidance for developing systems that align recommendations with user capabilities, enabling more usable and context-aware decision support in data science environments.

Automation frameworks support data science projects by automating tasks such as data preprocessing, feature engineering, model selection, and deployment. These frameworks reduce manual workload but create challenges for maintaining trustworthy, fair, and responsible analytics. Trustworthy analytics require human competencies to ensure explainability, fairness auditing, and ethical compliance, particularly where automation alone is insufficient. Existing taxonomies classify frameworks by technical scope but ignore the human competencies necessary for critical oversight. This paper addresses the gap by proposing a Competency-Aware Human-in-the-Loop (CA-HIL) Framework. The CA-HIL Framework systematically maps automation stages to intervention points, defines the associated risks, and identifies the specific human competencies required to mitigate these risks. Following the PRISMA methodology, a systematic literature review (SLR) of 2241 research articles supports the framework's development. The evaluation applies CA-HIL to leading automation systems such as TPOT, Auto-Prep, and AutoDS to identify in-dispensable human interventions. The findings demonstrate how competency-aware intervention enhances transparency, fairness, and accountability in automated workflows. This study provides a structured, evidence-based approach for designing responsible data science automation systems that bridge technical efficiency with human ethical responsibility.

A known technique to enable inexperienced users to apply sophisticated machine learning models is Automated Machine Learning (AutoML). AutoML can be used to determine appropriate algorithms and hyperparameters to specific problem settings. Typically, Hyperparameter Optimization (HPO) results in hyperparameters of a single algorithm, optimizing a given objective function. In the domain of automated clustering, the problem inherently has a non-unique solution, often yielding multiple valid algorithms. This multiplicity arises due to the exis-tence of various clustering algorithms, each employing different heuristics and optimization strategies as well as the subjectivity of the clustering notion. Consequently, the search space for finding an optimal clustering solution is highly complex, comprising the selection and parameter tuning of multiple algorithms. In this paper, we introduce the problem of Multi Algorithm Selection, which aims to find an appropriate set of algorithms of unknown size along with their respective hyperparameters, for automated clustering. To this end, we propose various search space strategies and utilize them as inputs for several well-known HPO algorithms to determine an optimal set of clustering algorithms. The resulting pipelines are evaluated on a number of publicly available, synthetic datasets to assess the ability of our methods to find interesting clustering sets. The strategy used to model a complex search space significantly influences the quality of the resulting model. Furthermore, clustering sets generated by means of our proposal yield higher quality than those created using naive approaches.

In the fast-evolving field of data science, the combination of the right team competencies has a major impact on a successful project execution. These competencies, ranging from data acquisition to model deployment, are increasingly difficult to maintain due to widespread competency shortages. This puts data science projects at risk of delays, inefficiencies, and failure, as organizations struggle to find skilled professionals. Automation frameworks - software tools designed to automate repetitive or complex tasks - offer a solution to this challenge. While these frameworks provide benefits such as reducing manual labor and improving project efficiency, they have notable limitations, particularly in covering critical phases like business understanding and deployment. Additionally, training programs also struggle to fully address the competency gap due to time, cost and scalability constraints. This paper investigates how existing automation frameworks can fill these competency gaps within data science teams more effectively. Using the CRISP-DM model as an example of a structured process, this study first identifies tasks required in each phase. Then, it matches these tasks with relevant automation frameworks to assess the extent of automation possible. Finally, these tasks are mapped to the EDISON Data Science Competence Framework to highlight which competencies automation frameworks can address. The findings suggest that automation frameworks effectively bridge competency gaps, enabling teams to complete projects more efficiently and effectively where human expertise may be lacking. In this manner, our findings serve as a reference point for data scientists and practitioners alike.

Big data technologies have been applied successfully to diverse application domains in order to facilitate analytics of voluminous and heterogeneous databases at scale. Digital sensory typically provides high-content data comprising multiple data recordings with high frequency. One example of such sensory are multi-electrode arrays (MEA), which are able to measure electric cell activity with high spatial and temporal resolution. The resulting multivariate time series and their inherent subsequences can then be analyzed and compared in aspects of time, space and shape. This analytical process is frequently performed manually by domain experts in combination with data analytical methods that help to identify and track signal beginnings, signal ends and signal propagations.In this paper, we propose an unsupervised approach to discover propagating signals in high-content, multivariate time series databases. To this end, we introduce an efficient spatio-temporal subsequence clustering algorithm that detects and tracks spatial and temporal signal progagations by means of density-based clusters. We present a formal propagation model and show how to adapt the DBSCAN algorithm to our specific application setting on pharmacological data. Our empirical investigation shows that our proposal is able to detect signal propagations with high accuracy and efficiency. Our approach hence scales not only to pharmacological settings but also to other biological, medical, and chemical domains making use of high-content multi-electrode array data.

Academic libraries are increasingly challenged by the need to efficiently manage and analyse vast collections of data and knowledge. The divers formats and organisation methods of these collections, ranging from traditional print media to digital archives and multimedia assets, can hinder researchers’ ability to easily access and retrieve relevant information. This paper introduces an ontology-driven knowledge base to address this issue by enabling the efficient access to knowledge in the application domain and enhancing the semantic search capabilities in the field of Digital Humanities. Our approach focuses on the development of an ontology-drive knowledge base for semantic search in academic libraries by the example of the library of the Folkwang University of Arts that captures the knowledge concepts present in the library’s archival collections. The resulting ontology framework provides a structured representation of domain knowledge, facilitating the integration of diverse data sources, including structured, semi-structured, and unstructured data from the application domain into a triple store knowledge base. By leveraging SPARQL queries generated from Large Language Model (LLM) prompts, we aim to facilitate more intuitive and effective knowledge retrieval. This approach allows users to express their information needs in a more natural and flexible way, leading to more accurate and relevant search results. We evaluate the proposed ontology-driven knowledge base in terms of its integrity, consistency, flexibility, relevance, and scalability. Our evaluation methodology includes a combination of verification and validation techniques, including automated reasoners and query results based on competence questions. Our findings demonstrate the potential of ontology engineering to enhance complex information retrieval in academic libraries. However, we also identify limitations related to processing speed for complex queries and the quality of search results. This research contributes to the field of computational archival science by providing a novel approach to semantic search in academic libraries. By enabling more precise and efficient access to knowledge, our ontology-driven knowledge base has the potential to enrich the academic and Digital Humanities landscape, empowering researchers to delve deeper into the vast resources available within these institutions.

The analysis of big datasets is a challenging task. While many data scientists are working in the field of supervised data analysis, there is also a growing demand in the field of unsupervised data analysis, such as clustering. To come up with a solution for this, multiple AutoML approaches for clustering have been proposed. However, most of these approaches try to find the "best" clustering, ignoring the subjective nature of the clustering task. A domain expert, however, might be able to identify an appropriate clustering for his/her application in a small set of clusterings, which have been generated, even if he/she is not capable of creating these clusterings by themselves. To enable domain experts to identify valuable clusterings without becoming an expert in clustering as well, we propose to generate multiple clusterings via AutoML processes and to return a selection of clusterings, from which the user can select the most preferred one. We will investigate the use of the Skyline Operator in this use case, to prune clusterings, which are likely useless, and to find a number of clusterings, which are usable for domain experts. We will investigate, how many clusters can be pruned this way and how many valuable clusters get falsely pruned. Our empirical investigation is carried out on a number of synthetic datasets, where a known ground truth can proxy for the wishes of a domain expert and multiple properties of the clusterings can be known beforehand.

Clustering is a frequently encountered task in big data analytics, where the goal is to simultaneously group and separate similar and dissimilar objects, respectively. It is also a well known fact, that clustering has a highly subjective nature, in the sense that determining the best clustering is highly dependent on the application setting. Though the recently established research direction of Automated Clustering has originated different algorithmic solutions to the clustering problem, these approaches assume a defined clustering evaluation metric to be optimized. These approaches thus inherently assume that such a thing like a single best clustering exists, which is not always true in real applications where insight into the data comes when inspecting the resulting clusterings.In order to maximize the insight for a data scientists or a domain specialist, we propose to not solely investigate a single best clustering but instead to explore multiple best clusterings according to different evaluation criteria. This will not only help to identify several clusters of interest to the user, but also to maximize the impact gained from following different evaluation criteria. In this paper, we hence propose the concept of Automated Exploratory Clustering, which follows the idea of automatically providing the best clusterings for further exploration. To this end, we formalize the problem of Automated Exploratory Clustering and define a theoretic framework comprising necessary formulations. In addition, we propose an efficient algorithm to compute the most interesting clusterings and benchmark its effectiveness and efficiency. Our approach will help domain experts without expertise in clustering to gain new insights in their datasets and serves as a baseline for future research.

Time series are able to depict various kinds of real-world processes, which are frequently represented by the aggregation of multiple real-world components acting in parallel. Extracting these components and using them for discovering structural regularities is a major challenge in the domain of time series analytics. In this paper, we propose the CATGP+ algorithm for efficient Component Analysis in Time series with Gaussian Processes. For this purpose, we assume time series to be modelled by Gaussian processes, which are probabilistic machine learning models that are able to capture local linearity, trend, periodicity, etc. Based on these models, the proposed CATGP+ algorithm facilitates the determination of structural regularities, i.e. frequently occurring components, within Gaussian processes. By relating the components of the Gaussian processes, modelling the underlying time series data, with those appearing in real-world processes, we are able to gain further inside into behavior and dependencies on a structural and substructural level. Moreover, as our designed solution is compatible with the classical itemset mining problem, we are able to inherent the efficiency of existing algorithmic approaches. Our experimental evaluation indicates that the CATGP+ algorithm is able to efficiently discover frequent components hidden in the underlying time series data.

Clustering is a fundamental method for advanced data analytics. Not only the selection of a suitable clustering method, but also the choice of the resulting clustering, which complies with the application requirements and the data-analytical hypotheses, is a challenge for complex analysis settings. While there exists a multitude of different clustering algorithms for the computation of simple convex up to arbitrary shaped clusters, the question of how to quantify the quality of each individual clustering remains a challenge. In this paper, we investigate the ability of state-of-the-art Clustering Validation Indices (CVI) to assess clustering performance. To this end, we provide a survey of the inner workings of the different CVI and an extensive benchmark on 180 publicly available datasets. Furthermore, we evaluate both the Euclidean distance and the density-based DC-distance to quantify the quality of arbitrary shaped clusters. Our performance evaluation indicates that no singular CVI performs significantly better than the others in general and that the density-based DC-distance is well suited for finding arbitrary shaped clusters even with CVI not specifically designed for this task. Moreover, we discovered that no single CVI effectively performs well for both arbitrary shaped and overlapping clusters at the same time. Our survey provides a comprehensive analysis of CVI from both a theoretical and a practical point of view, and is thus a useful guideline for researchers and practitioners in academia and business.

The project Knowledge Graphs for competency-based Education (KG4CBE) conducts educational data science research to establish competency-based instruction in higher-education programs. In this paper, we propose the design of a knowledge graph to examine the impact of instructional design on student-teacher interaction to facilitate complex learning. For this purpose, the knowledge graph will incorporate the components of an online introductory data science course, including educational materials and learning tasks created with the Four Component Instructional Design (4C/ID) model. Furthermore, the knowledge graph will incorporate data recording the behaviors, interactions and assessments of participating students. To study the competency-based instruction process, the proposed knowledge graph must be scalable to the big data quantities common in educational settings. Therefore, the knowledge graph will be deployed as a tool with accompanying routines to acquire, simulate and load educational data. Furthermore, this tool will provide methods to interact with and visualize the stored information. As future research, we aim to evaluate the proposed knowledge graph in a large-scale educational design research study, to examine the impact of monitoring, forecasting and recommendations in complex learning settings.

Automated Machine Learning enables many inexperienced users to generate good classification solutions solely by letting machines generate optimal pipelines. However, there are not as many possibilities to generate meaningful clusterings without further knowledge. As the clustering problem is context and domain dependent, a single solution can never be the sole best clustering for any dataset. To overcome this problem, in this paper we design a framework which uses clustering algorithms, CVIs, the skyline operator and multiple visualization techniques to generate diverse interesting clusterings and present them to a user, enabling him to take an informed decision without needing any experience in the field of clustering.

Many datasets are of increasingly high dimension- ality, where a large number of features could be irrelevant to the learning task. The inclusion of such features would not only introduce unwanted noise but also increase computational complexity. Deep neural networks (DNNs) outperform machine learning (ML) algorithms in a variety of applications due to their effectiveness in modelling complex problems and handling high-dimensional datasets. However, due to non-linearity and higher-order feature interactions, DNN models are unavoidably opaque, making them black-box methods. In contrast, an interpretable model can identify statistically significant features and explain the way they affect the model’s outcome. In this paper, we propose a novel method to improve the interpretability of blackbox models in the case of high-dimensional datasets. First, a black-box model is trained on full feature space that learns useful embeddings on which the classification is performed. To decompose the inner principles of the black-box and to identify top-k important features (global explainability), probing and perturbing techniques are applied. An interpretable surrogate model is then trained on top-k feature space to approximate the black-box. Finally, decision rules and counterfactuals are derived from the surrogate to provide local decisions. Our approach outperforms tabular learners, e.g., TabNet and XGboost, and SHAP-based interpretability techniques, when tested on a number of datasets having dimensionality between 54 and 20,5311.1GitHub: https://github.com/rezacsedu/DeepExplainHidim

When multiple sensors record spatially proximate areas of activity, spreading activity patterns appear as temporally shifted signals in multiple time series. This is particularly prominent in the domains of medical and health analysis, where multi-sensory data is the object of time-elastic investigation. Tracing the spread of these patterns still remains a challenge in time series analysis. In this paper, we propose Motif Tracking for Spatially Ordered Time Series (MoTrack), an algorithm to efficiently track the propagation of individual patterns of activity throughout spatially ordered time series. Additionally, we present the concept of propagation trees to represent this propagation for a given point of origin. We investigate our proposal by applying MoTrack to high-frequency recordings of the electrical activity of β-cells located inside the pancreatic islet. The results confirm MoTrack’s capability to trace dynamically evolving signals in such recordings and indicate that future work using this approach can address current challenges in diabetes research.

Gaussian Processes offer non-parametric, probabilistic models that can be used in numerous fields of data analysis. One major drawback is their lack of adjustability in case of drifting and evolving streaming data, where inherent kernels need to be adapted in an efficient manner. To counteract this issue, we propose a novel automated kernel search method that allows us to incrementally adapt Gaussian Process models to evolving IoT data streams. Our approach, denoted as Adjusting Kernel Search (AKS), offers an efficient alternative to searching for suitable kernels from scratch. We evaluate the AKS algorithm on several IoT datasets and show that our approach is able to achieve higher accuracy with lower run-times compared to previous approaches.

Time series data is one of the complex data types commonly encountered in many application areas ranging from automotive, finance, medicine to industry. A prominent task is time series classification, which entails identifying expressive features in oder to predict class labels of time series data. In this paper, we propose a novel approach for time series classification called Local Gaussian Process Model Inference Classification (LOGIC). Our concept consists in (i) learning latent characteristics of given time series data by means of Gaussian processes, (ii) using these characteristics to embed time series into a more expressive feature space and (iii) classifying time series data based on these features via existing classification methods. By making use of various general-purpose classification methods, we show that LOGIC is able to compete with state-of-the-art approaches in terms of accuracy and efficiency.

Trends, periodicities and local variations are among the main recognizable patterns in time series data. While humans are able to quickly explore the superimposition of such patterns, mining algorithms are often faced with the challenge of finding (i) a suitable time series representation model and (ii) an expressive query model which adapt to diverse application domains and information needs. In this paper, we propose a supervised stochastic approach which facilitates interval-based pattern analysis of time series data. Our proposal is based on non-parametric Gaussian Processes and is able to interrelate interesting patterns within single and across multiple time series. Our performance evaluation in different real-world application domains indicates that our approach is able to expose interesting patterns and knowledge.

A flexible, domain-agnostic function approximator, which is robust towards unreliable, noisy and partially missing data, would be an ideal tool for pattern mining in large-scale data. Although Gaussian Process Models (GPMs), which are widely regarded as a probabilistic tool for capturing inherent data characteristics, satisfy those requirements, full Gaussian Process inference and training is limited to a few thousand data records. Moreover, a process of automatic GPM inference is required to find an optimal model for a given dataset, despite prevailing default instantiations and existing prior knowledge in some scenarios, which both shortcut the way to an optimal GPM. Since non-approximate Gaussian Processes only allow for processing small datasets with low local statistical versatility, we propose a new approach that enables to automatically infer GPMs of adaptive local complexity on large scale multivariate data. The resulting model is composed of independent statistical representations for disjoint partitions varying in statistical versatility. Our performance evaluation indicates an improvement in inference runtime, while maintaining high model quality with regards to state-of-the-art GPM inference algorithms.

One of the prominent types of time series analytics is classification, which entails identifying expressive class-wise features for determining class labels of time series data. In this paper, we propose a novel approach for time series classification called Local Gaussian Process Model Inference Classification (LOGIC). Our idea consists in (i) approximating the latent, class-wise characteristics of given time series data by means of Gaussian processes and (ii) aggregating these characteristics into a feature representation to (iii) provide a model-agnostic interface for state-of-the-art feature classification mechanisms. By making use of a fully-connected neural network as classification model, we show that the LOGIC model is able to compete with state-of-the-art approaches.

Gaussian Process Models (GPMs) are widely regarded as a prominent tool for capturing the inherent characteristics of data. These bayesian machine learning models allow for data analysis tasks such as regression and classification. Usually a process of automatic GPM retrieval is needed to find an optimal model for a given dataset, despite prevailing default instantiations and existing prior knowledge in some scenarios, which both shortcut the way to an optimal GPM. Since non-approximative Gaussian Processes only allow for processing small datasets with low statistical versatility, we propose a new approach that allows to efficiently and automatically retrieve GPMs for large-scale data. The resulting model is composed of independent statistical representations for non-overlapping segments of the given data. Our performance evaluation of the new approach demonstrates the quality of resulting models, which clearly outperform default GPM instantiations, while maintaining reasonable model training time.

275

One of the key-enabling technologies in our digital information era is the Internet of Things (IoT). It provides a multitude of data technologies and analysis methodologies in order to connect internet-enabled devices and to collect, exchange, and analyze data snippets and information assets. It is forecasted that the majority of real-time data will be generated from devices interconnected within the Internet of Things. In order to be able to manage and access IoT data efficiently, we propose to make use of ptolemaic indexing. This domain-agnostic approach is applicable to many data-intensive IoT environments including a wide range of data types and (dis)similarity measures. In this paper, we provide an introduction to metric and ptolemaic indexing and evaluate their performance on different large-scale IoT datasets. The results of our performance evaluation indicate the high potential of our proposal for indexing and searching large-scale IoT data efficiently.

Complex, multivariate data streams frequently comprise subjacent behavioral patterns, which are subsumable by a process of statistical structure discovery. Revealing these hidden patterns from raw data is a major challenge in abstracting information and thus for new opportunities of efficient data analysis at scale. State-of-the-art approaches, such as CKS and ABCD, leverage statistical data models and Gaussian Processes in order to abstract from raw data and to describe their major data characteristics by means of kernel-decomposed covariance functions. The process of identifying the most appropriate covariance function is a performance bottleneck due to its super-quadratic computation time complexity for model selection and evaluation. In this paper, we thus propose a new approach for the computation of large-scale statistical data models. To this end, we propose to bound the complexity of the statistical data model and develop a sequential agglomerative approach to reduce the computational load of the required evaluative calculations. Our performance analysis indicates that our proposal is able to outperform state-of-the-art kernel search algorithms such as CKS and ABCD with respect to the qualities of efficiency and accuracy.

In this work we analyze content statistics of the V3C1 dataset, which is the first partition of theVimeo Creative Commons Collection (V3C). The dataset has been designed to represent true web videos in the wild, with good visual quality and diverse content characteristics, and will serve as evaluation basis for the Video Browser Showdown 2019-2021 and TREC Video Retrieval (TRECVID) Ad-Hoc Video Search tasks 2019-2021. The dataset comes with a shot segmentation (around 1 million shots) for which we analyze content specifics and statistics. Our research shows that the content of V3C1 is very diverse, has no predominant characteristics and provides a low self-similarity. Thus it is very well suited for video retrieval evaluations as well as for participants of TRECVID AVS or the VBS.

Given a very large multimedia database, how to process k-nearest-neighbor queries efficiently? While the sequential scan is one of the most obvious solutions for small-to-moderate multimedia databases, it becomes practically infeasible when the database size grows. Concomitant with the volume and velocity of data, multimedia databases are frequently endowed with a complex distance-based similarity model that supports content-based data access in an adjustable and adaptive manner. Typical for many state-of-the-art distance-based similarity models is an at least quadratic computation time complexity for a single distance evaluation between two multimedia objects. Thus the search for the most query-like multimedia objects is still one of the major challenges.In this paper, we address the problem of optimal k-nearest-neighbor query processing via multiple lower bound approximations in very large multimedia databases. To this end, we propose the concepts of filter-optimality and refinement-optimality and present the Cascading Multi-Step Algorithm and the Interleaved Multi-Step Algorithm for fast query processing. Besides the algorithms' properties, we study their query processing performance with respect to the number of CPU and I/O operations on large-scale benchmark multimedia databases. Our performance analysis shows how to process k-nearest-neighbor queries in multimedia databases efficiently and provides a guide for further research.

Data are a central phenomenon in our digital information age. They impact the way we live, work, and play and provide unprecedented opportunities to simplify our daily life and behavior. They implicate enormous potential and impact society, economy, and science. Due to the advancement of cyber-physical systems and Internet of Things technologies, it is expected that the majority of real-time data will be generated from devices interconnected within the Internet of Things by the year 2025. In this paper, we tackle the problem of managing Internet of Things data in an efficient way. To this end, we introduce the metric approach for storing and querying Internet of Things data and investigate the ability of pivot-based tables for indexing and searching this type of data. Along with the introduction of two real-world, large-scale Internet of Things datasets from the EU projects COMPOSITION and MONSOON (under grant no. 723145 and 723650), we show that the metric approach facilitates efficient data access in the Internet of Things.

Accessing very large multimedia databases in a content-based way has become one of the major challenges in todays' multimedia analysis and retrieval applications. Accompanied by the heterogeneity of data and the continuous change of user requirements, content-based approaches are supposed to efficiently retrieve and analyze query-like multimedia objects with the highest possible degree of efficacy, which makes the utilization of complex multimedia object representations and adaptive similarity measures inevitable. In order to facilitate efficient and flexible content-based access into multimedia databases comprising millions of complex data objects, we propose the Multi-step Threshold Algorithm (MTA). Based on a set of query features, the MTA aims to retrieve the most similar multimedia objects with minimal I/O cost by incrementally traversing an in-memory index structure in a feature-by-feature manner in order to approximate the multimedia objects' similarities prior to database access. In addition to the MTA, we propose different enhancements that ensure scalable feature-based query processing. Our performance analysis evidences that our proposal is able to process feature-based queries on a million-scale multimedia database in milliseconds on a single CPU.

This tutorial aims at providing a unified and comprehensive overview of the state-of-the-art approaches to distance-based multimedia indexing.

The rapid increase in generation and dissemination of online video data has recently raised the demand on efficient and effective query processing techniques in large video databases. In this paper, we first introduce a novel compact video representation model to achieve high effectiveness, and then propose to alleviate computational time complexity of the well-known Earth Mover’s Distance by introducing a filter approximation analyzing earth flows locally and restricting the number of flows globally, ensuring completeness .M oreover, extensive experimental evaluation performed on high dimensional real world datasets points out high efficiency and effectiveness of the proposals, significantly reducing the number of Earth Mover’s Distance computations and outperforming the state of the art by up to two orders of magnitude with respect to selectivity and query processing time.

With the continuous increase of heterogeneous multimedia data, the question of how to access big multimedia data efficiently has become of crucial importance. In order to provide fast access to complex multimedia data, we propose to approximate content-based features of multimedia objects by means of generative models. The proposed gradient-based signatures epitomize a high quality content-based approximation of multimedia objects and facilitate efficient indexing and query processing at large scale.

With the continuous rise of multimedia, the question of how to access large-scale multimedia databases efficiently has become of crucial importance. Given a multimedia database comprising millions of multimedia objects, how to approximate the content-based properties of the corresponding feature representations in order to carry out similarity search efficiently and with high accuracy? In this paper, we propose the concept of gradient-based signatures in order to aggregate content-based features of multimedia objects by means of generative models. We provide theoretical insights into our approach including closed-form expressions for the computation of gradient-based signatures with respect to Gaussian mixture models and additionally investigate different binarization methods for gradient-based signatures in order to query databases comprising millions of multimedia objects with high accuracy in less than one second.

The recent rapid growth of scientific data necessitates efficient similarity search techniques for which convenient object representation models are of vital importance. Feature signatures denoting highly flexible object feature representations have increasingly gained attention for which corresponding efficiency improvement techniques are developed. In this paper, we focus on efficient query processing with the well-known Earth Mover's Distance (EMD) on databases of feature signatures, and propose efficient approximation techniques successfully applicable to high-dimensional feature signatures via dimensionality reduction, guaranteeing both completeness and no false-dismissal within a filter-and-refine architecture. Rigorous experiments on real world data indicate a considerable reduction in the number of EMD computations and high efficiency of the proposed techniques which significantly reduce the query processing time.

The Earth Mover's Distance, proposed in computer vision as a distance-based similarity model reflecting the human perceptual similarity, has been widely utilized in numerous domains for similarity search applicable on both feature histograms and signatures. While efficiency improvement methods towards the Earth Mover's Distance were frequently investigated on feature histograms, not much work is known to study this similarity model on feature signatures denoting object-specific feature representations. Given a very large multimedia database of features signatures, how can k-nearest-neighbor queries be processed efficiently by using the Earth Mover's Distance? In this paper, we propose an efficient filter approximation technique to lower bound the Earth Mover's Distance on feature signatures by restricting the number of earth flows locally. Extensive experiments on real world data indicate the high efficiency of the proposal, attaining order-of-magnitude query processing time cost reduction for high dimensional feature signatures.

We propose a simple yet effective approach to content-based image retrieval: the signature matching distance. While recent approaches to content-based image retrieval utilize the bag-of-visual-words model, where image descriptors are matched through a common visual vocabulary, signature-based approaches use a distance between signatures, i.e. between image-specific bags of locally aggregated descriptors, in order to quantify image dissimilarity. In this paper, we focus on the signature-based approach to content-based image retrieval and propose a novel distance function, the signature matching distance. This distance matches coincident visual properties of images based on their signatures. In particular, by investigating different descriptor matching strategies and their suitability to match signatures, we show that our approach is able to outperform other signature-based approaches to content-based image retrieval. Moreover, in combination with a simple color and texture-based image descriptor, our approach is able to compete with the majority of bag-of-visual-words approaches.

The Signature Quadratic Form Distance on feature signatures represents a flexible distance-based similarity model for effective content-based multimedia retrieval. Although metric indexing approaches are able to speed up query processing by two orders of magnitude, their applicability to large-scale multimedia databases containing billions of images is still a challenging issue. In this paper, we propose the utilization of GPUs for efficient query processing with the Signature Quadratic Form Distance. We show how to process multiple distance computations in parallel and demonstrate efficient query processing by comparing many-core GPU with multi-core CPU implementations.

The Signature Quadratic Form Distance has been introduced as an adaptive similarity measure coping with flexible content representations of various multimedia data. Although the Signature Quadratic Form Distance has shown good retrieval performance with respect to their qualities of effectiveness and efficiency, its applicability to index structures remains a challenging issue due to its dynamic nature. In this paper, we investigate the indexability of the Signature Quadratic Form Distance regarding metric access methods. We show how the distance's inherent parameters determine the indexability and analyze the relationship between effectiveness and efficiency on numerous image databases.

We introduce a distance-based similarity model with application to the optimization of the fuel injection process. Our model allows for an automatic evaluation of huge and complex amount of experimental data originated from optical measurement techniques analyzing the fuel injection process. The goal is to enable researchers to get deeper insight into this process based on an automatically driven analysis.

The Earth Mover's Distance is a well-known distance measure employed in various domains, especially for content-based retrieval in multimedia databases. However, the distance evaluation is a considerably expensive task and thus for large multimedia databases, efficient query processing becomes a challenging problem. In this paper, we introduce a parameterized version of the Earth Mover's Distance that can be used by database experts to change the distance distribution in the derived distance space in order to improve the indexability. We empirically show, that we can significantly improve the indexability of the distance space and that we can tune the retrieval quality by adapting the parameterized Earth Mover's Distance.

The signature quadratic form distance has been introduced as an adaptive similarity measure coping with flexible content representations of multimedia data. While this distance has shown high retrieval quality, its high computational complexity underscores the need for efficient search methods. Recent research has shown that a huge improvement in search efficiency is achieved when using metric indexing. In this paper, we analyze the applicability of Ptolemaic indexing to the signature quadratic form distance. We show that it is a Ptolemaic metric and present an application of Ptolemaic pivot tables to image databases, resolving queries nearly four times as fast as the state-of-the-art metric solution, and up to 300 times as fast as sequential scan.

Determining similar objects is a fundamental operation both in data mining tasks such as clustering and in query-driven object retrieval. By definition of similarity search, query objects can only be imprecise descriptions of what users are looking for in a database, and even high-quality similarity measures can only be approximations of the users' notion of similarity. To overcome these shortcomings, iterative query refinement systems have been proposed. They utilize user feedback regarding the relevance of intermediate results to adapt the query object and/or the similarity measure.