The importance of exploiting profiling metadata, such as Relaxed Functional Dependencies (RFDs), to support advanced data processing tasks, continues to grow also due to the availability of algorithms capable of automatically extracting them from data. Nevertheless, in order to use this type of metadata in real-life contexts, it is also necessary to ensure their correct interpretation of their meaningfulness and their possible evolution over time. To this end, in this paper, we present a new tool that allows visual analysis and explainability of how discovery results evolve according to changes in the data. More specifically, it provides a comprehensive overview of the impact that data changes, by possibly analyzing in-depth affected dependencies and understanding motivations underlying their evolution through a textual explanation. The effectiveness of the proposed tool has been evaluated by conducting a user study, which highlighted RYAN’s capability to yield an intuitive visualization of RFD discovery results and to provide a clear explanation of the reasons that led to the evolution of RFDs.

Over the past decade, there have been numerous extensions to the definition of Functional Dependency (fd), culminating in the introduction of Relaxed Functional Dependency (rfd), offering more flexible constraints compared to traditionalfds. This increased flexibility makesrfds well-suited for exploring and profiling data in datasets with lower data quality. However, efficiently identifyingrfds within dynamic data sources presents a significant challenge, as it requires processing an entire dataset from scratch whenever modifications occur. To tackle this problem, incremental discovery algorithms have been defined, but they often suffer when the frequency and the size of batches of updates increase. This article presents a new algorithm, namelyD-IndiBits, relying on a new decentralized architecture to balance the workload that drives the incremental discovery process ofIndiBits, which is based on bitwise operators for computing attribute similarities. Experiments demonstrateD-IndiBits's effectiveness compared tofdandrfddiscovery algorithms on both static and dynamic real-world data. With batches of modifications of sizes 10 k and 100 k,D-IndiBitsis capable of updating the set ofrfds in a few seconds, whereas all other approaches often employ more than 3 hours.

One of the main challenges in data profiling is to efficiently extract metadata from dynamic information sources, by avoiding the processing of the whole dataset from scratch upon modifications. In this paper, we present IndiBits, an algorithm for discovering relaxed functional dependencies (RFDs for short), which represent data relationships relying on approximate matching paradigms. IndiBits is able to dynamically infer and update the RFDs holding on a dataset upon modification operations performed on it. It exploits a binary representation of data similarities, a new validation method, and specific search methods, to dynamically update the set of RFDs, based on previously holding RFDs and the type of modifications performed over data. Experimental results demonstrate the effectiveness of IndiBits on real-world datasets, even in comparison with FD and RFD discovery algorithms in both static and dynamic scenarios.

To assess the quality of data it is useful to extract properties and relationships among them. However, exceptions and approximations need be considered in real-world settings. To this end, relaxed FDs (RFDs) are data dependencies accounting for both exceptions and similarities on data, but their discovery is an extremely complex problem, also due to the necessity of specifying similarity and validity thresholds. The RFD discovery algorithm presented in this paper exploits the concept of dominance to automatically derive similarity thresholds. The discovery performances and the effectiveness of the proposed algorithm are assessed through a comparative evaluation with state-of-art approaches.

With the advent of Big Data there is an increasing necessity to incrementally mine information from data originating from sensors and other dynamic sources. Thus, it is necessary to devise algorithms capable of mining useful information upon possible evolutions of databases. Among these, there are certainly data profiling info, such as functional dependencies (fd for short), which are particularly useful for data integration and for assessing the quality of data. The incremental scenario requires the definition of search strategies and validation methods able to analyze only the portion of the dataset affected by the last changes. In this paper, we propose a new validation method, which exploits regular expressions and compressed data structures to efficiently verify whether a candidate fd holds on an updated version of the dataset. Experimental results demonstrate the effectiveness of the proposed method on real-world datasets adapted for incremental scenarios, also compared with a baseline incremental fd discovery algorithm.

With the advent of big data and data lakes, data are often integrated from multiple sources. Such integrated data are often of poor quality, due to inconsistencies, errors, and so forth. One way to check the quality of data is to infer functional dependencies (fds). However, in many modern applications it might be necessary to extract properties and relationships that are not captured through fds, due to the necessity to admit exceptions, or to consider similarity rather than equality of data values. Relaxed fds (rfds) have been introduced to meet these needs, but their discovery from data adds further complexity to an already complex problem, also due to the necessity of specifying similarity and validity thresholds. We propose Domino, a new discovery algorithm for rfds that exploits the concept of dominance in order to derive similarity thresholds of attribute values while inferring rfds. An experimental evaluation on real datasets demonstrates the discovery performance and the effectiveness of the proposed algorithm.

Nowadays, the human influence often depends on the number of followers that an individual has in his/her own social media context. To this end, the presence of fake accounts is one of the most relevant problems and can potentially have a big impact on many real life and business activities. Fake followers are dangerous for social platforms, since they may alter concepts like popularity and influence, which might yield a strong impact on economy, politics, and society. Thus, it is necessary to devise new methodologies enabling the possibility to identify and characterize fake accounts. This work presents a novel technique to discriminate real accounts on social networks from fake ones. The technique exploits knowledge automatically extracted from big data to characterize typical patterns of fake accounts. We empirically evaluated the proposed technique on the Twitter social network, and achieved significant results in terms of discrimination capabilities.

Functional dependencies (fds) express important relationships among data, which can be used for several goals, including schema normalization and data cleansing. However, to solve several issues in emerging application domains, such as the identification of data inconsistencies or patterns of semantically related data, it has been necessary to relax the fd definition through the introduction of approximations in data comparison and/or validity. Moreover, while fds were originally specified at design time, with the availability of massive data and computational power many algorithms have been devised to automatically discover them from data, including algorithms for discovering some types of relaxed fds. In this paper we present a technique that exploits lattice-based algorithms for the discovery of fds from data, in order to detect relaxed fds. Moreover, we introduce an algorithm to determine a proper distance threshold for a given relaxed fd holding over the entire database.

Recently, there has been a renovated interest in functional dependencies due to the possibility of employing them in several advanced database operations, such as data cleaning, query relaxation, record matching, and so forth. In particular, the constraints defined for canonical functional dependencies have been relaxed to capture inconsistencies in real data, patterns of semantically related data, or semantic relationships in complex data types. In this paper, we have surveyed 35 of such functional dependencies, providing a classification criteria, motivating examples, and a systematic analysis of them.

One of the problems arising upon the evolution of a database schema is that some queries and views defined on the previous schema version might no longer work properly. Thus, evolving a database schema entails the redefinition of queries and views to adapt them to the new schema. Although this problem has been mainly raised in the context of traditional information systems, solutions to it are also advocated in other database-related areas, such as Data Integration, Web Data Integration, and Data Warehouses. The problem is a critical one, since industrial organizations often need to adapt their databases and data warehouses to frequent changes in the real world. In this article, we provide a survey of existing approaches and tools to the problem of adapting queries and views upon a database schema evolution; we also propose a classification framework to enable a uniform comparison method among many heterogeneous approaches and tools.

Despite many innovative systems supporting the data integration process, designers advocate more abstract metaphors to master the inherent complexity of this activity. In fact, the visual notations provided in many modern data integration systems might run into scale up problems when facing the integration of big data sources. Thus, higher level visual notations and automatic schema mapping mechanisms might be the key factors to make the data integration process more tractable. In this paper we present the Conceptual Data Integration Language (CoDIL), a visual language providing conceptual level visual mechanisms to manipulate and integrate data sources, together with a formalization of the language icon operators by means of ALCN Description Logic. The formalization allowed us to define the logic-level semantics of CoDIL, providing reasoning rules for validating the correctness of a data integration process and for generating the logic-level reconciled schema.