The incorporation of domain knowledge (DK) in AI has been studied for years and turns out to be critical in practice. Functions are also a basic notion for dealing with data science projects and are somehow related to DK. Consider the following scenario. Let \(D(y, z_1, \ldots , z_n)\) be a dataset, Alice a data scientist, Bob a domain expert and \(y = f(z_1, \ldots , z_n)\) a function known to Bob from his background knowledge. Alice is interested in the following simple yet crucial questions: How to define the satisfaction of f in D ? How to measure that satisfaction efficiently? How does this satisfaction relate to the supervised learning task of learning f from D ? It turns out that these problems are related to the study of counterexamples through the use of functional dependencies (FDs) and, in particular, FD measures used to quantify their satisfaction in a dataset such as the \(g_3\) indicator where the equality is replaced by more flexible predicates. In this article, we first examine the complexity of computing \(g_3\) . It is known that \(g_3\) can be computed in polynomial time when using equality, while it becomes NP -hard when using general predicates. Our goal is to refine this dichotomy by studying the impact of the following common properties: reflexivity, transitivity, symmetry, and antisymmetry. We show that symmetry and transitivity together are sufficient to guarantee that the \(g_3\) can be computed in polynomial time. However, removing one of them makes the problem NP -hard. Second, we study the computation of \(g_3\) in the polynomial and NP -hard cases identified previously. We propose different exact and approximate solutions for the computation of \(g_3\) in both cases. We compare these solutions in a detailed experimental study of time performance and approximation accuracy. All the algorithms are also made available via fastg3 , an open-source Python library with an underlying C++ implementation. Finally, we link counterexamples and \(g_3\) to supervised learning with a web application called adesit . adesit is intended to be part of an iterative data refinement process right after data selection and just before the machine learning process itself. It provides a way to evaluate the ability of a dataset to perform well for a given supervised learning problem through statistical and visual exploration. In a last section, we validate our approach by applying it to the industrial problem of air gap monitoring in compact hydro-generators.

Surrogate keys are now extensively utilized by database designers to implement keys in SQL tables. They are straightforward, easy to understand, enable efficient access, and are often considered a sufficient guarantee of data integrity despite lacking any real-world semantic meaning. In spite of all their benefits, one might wonder whether surrogate keys can negatively impact data quality. IT developers who rely exclusively on surrogate keys when designing database schemas may be tempted to not encode natural keys, as they are perceived as complex to manage at the application level. In such settings, surrogate keys allow the presence of so-called artificial unicity , a complex form of redundancy that can be propagated through foreign keys, and other underlying data-quality issues. In the presence of artificial unicity, most data cleaning techniques, especially unsupervised, are likely to fail, making data preparation and analytics very challenging. For relational databases implemented with surrogate keys but no natural keys, we developed RED2Hunt (RElational Databases REDundancy Hunting), a human-in-the-loop framework for identifying hidden redundancy and, if problems occur, clean the database. The framework was implemented on top of PostgreSQL within an eponym web-based platform to guide the expert through its application. In this paper, we present a demonstration of the RED2Hunt tool through three interactive scenarios on a polluted instance of the publicly available Perfect Pet database. During the demonstration, the visitor can take on one of two roles in the Perfect Pet database: a domain expert or a data scientist. As a domain expert, she will interact with RED2Hunt, for example to elicit natural keys, from simple yet very intuitive visualizations of tables' attributes. As a data scientist, she will explore two simple scenarios—executing SQL queries or applying learning models—on both the initial and cleaned databases to grasp the benefits of the approach.

In the context of process monitoring and predictive maintenance, an adapted visualisation of sensor data is essential in order to help the domain experts to make the right maintenance decision. The large volume and diversity of data leads us to aggregate the data to obtain semantically rich information useful to the domain expert. We study the case of industrial machinery equipped with several sensors producing time series, and we consider that this machinery has different operating states in its operation. We propose a method to identify an optimal representation of the data in 2 dimensions, understandable by the domain expert. This representation allows to easily identify the operating modes of the equipment and the possible deviation from a "normal" behavior. We use co-occurrence matrices to synthesise the time series data, and the features of interest and discretization are selected using two proposed criteria to measure the separation of working modes.

Taking domain knowledge into account is a long-standing issue in AI, especially nowadays where huge amounts of data are collected in the hope of delivering new in-sights and value. Let us consider the following scenario. Let D(y, x1, … ,xn) be a dataset, Alice a data scientist, Bob a domain expert and$y$=$f$(x1, … , xn) a function known by Bob from his background knowledge. We are interested in the following simple yet crucial questions for Alice: how to define the satisfaction of f in D and how difficult is it to measure that satisfaction? It turns out that those problems are related to functional dependencies (FDs) and especially FD measurements used to quantify their satisfaction in a dataset such as the g3 indicator. In this paper, we examine the computation of g3 with crisp FDs (aka. exact FDs) and a large class of non-crisp FDs replacing strict equality by more flexible predicates. Interestingly, it is known that the computation of g3 with crisp FDs is polynomial but turns out to be NP-Hard for non-crisp FDs. In this paper, we propose different exact and approximate solutions for the computation of g3 for both types. First, for crisp FDs with very large datasets, we propose solutions based on uniform and stratified random sampling. Second, for non-crisp FDs we present a detailed computation pipeline with various computation optimizations, including approximation algorithms and adaptations of recent developments in sublinear algorithms for NP-Hard problems. We also propose an in-depth experimental study of the algorithms presented in terms of time performances and approximation accuracy. All the algorithms are also made available through FASTG3, an open-source Python library designed to be intuitive and efficient thanks to an underlying C++ implementation.

Thanks to the numerous machine learning tools available to us nowadays, it is easier than ever to derive a model from a dataset in the frame of a supervised learning problem. However, when this model behaves poorly compared with an expected performance, the underlying question of the existence of such a model is often underlooked and one might just be tempted to try different parameters or just choose another model architecture. This is why the quality of the learning examples should be considered as early as possible as it acts as a go/no go signal for the following potentially costly learning process. With ADESIT, we provide a way to evaluate the ability of a dataset to perform well for a given supervised learning problem through statistics and visual exploration. Notably, we base our work on recent studies proposing the use of functional dependencies and specifically counterexample analysis to provide dataset cleanliness statistics but also a theoretical upper bound on the prediction accuracy directly linked to the problem settings (measurement uncertainty, expected generalization...). In brief, ADESIT is intended to be part of an iterative data refinement process right after data selection and right before the machine learning process itself. With further analysis for a given problem, the user can characterize, clean and export dynamically selected subsets, allowing to better understand what regions of the data could be refined and where the data precision must be improved by using, for example, new or more precise sensors.

This paper presents a virtual sensor for helicopter Main Gear Box (MGB) oil pressure. It is developed for anticipating failures of systems related to MGB lubrication. The virtual sensor is built using Machine Learning and massive in-service flight data collected from Airbus helicopters flying world-wide. The correlation between oil pressure values and other flight parameters is learnt during stable phases of flights in which the system is in its nominal state. At each flight, the values continuously estimated by the virtual sensor are compared to the measured ones, and an alert is raised when the difference becomes higher than a statistically predefined threshold. The virtual sensor was tested using normal and abnormal flights, and the results obtained so far in terms of anomaly detection performance are promising.

To help databases users who have just started learning SQL or are not familiar with their database, we propose ExplIQuE, an exploration interface with query extensions. Its purpose is to assist users to smoothly dive into data exploration, and to be able to express imprecise questions over their data. Indeed, such situations are more and more current with the increasing desire for users to get value out of their data. In this configuration, in addition to classic SQL querying possibilities, ExplIQuE offers the possibility to extend a given SQL query, by suggesting a set of possible selection predicates to add to the query, that aim at dividing the initial answer set to identify interesting exploration zones. In addition, ExplIQuE proposes some indicators to help the user in choosing its desire extension and in understanding her data, as well as interactive visualizations of the result set, in two dimensions revealed by PCA techniques. In this demonstration, we offer the audience the possibility to try the various functionalities of ExplIQuE by trying to express an imprecise question over a scientific database on bacterial colonies, through an iterative process. A video of the proposed demonstration is available at \urlhttps://youtu.be/oK8xWGCWj_A.

International audience

Semantic Query Optimization (SQO) dates back to the eighties and basically relies on integrity constraints (mainly functional dependencies) to achieve logical query optimization. Applied to temporal databases with integrity constraints given as temporal functional dependencies (TFDs), SQO turns out to be very difficult in the general case due to the time dimension. Motivated by an application on Intelligent buildings, we have defined a simple class of temporal queries for which SQO techniques apply. Given a temporal database, a set of TFD and a logical temporal query belonging to the class, we propose a new rewriting technique allowing to produce a more efficient query, equivalent to the initial query. Interestingly, the time dimension opens new opportunities for SQO. We apply our proposition to a scenario of Intelligent building (IBs). We point out that temporal databases and TFDs are well-suited for modeling sensor data in IB. We show how IB inhabitants' preferences on their life condition can be naturally represented with TFDs.

Nowadays, every company could understand how its business evolves from the data (deluge) generated by its activities. Roughly speaking, two types of data co-exist: historical data and real-time data from which business analysts have to take their decisions in a timely fashion. In this context, the notions of time (application time and transaction time) and traceability turn out to play a crucial role to understand what happened in the company and what is currently happening. Tornado offers a full-fledged platform to deal with such data and is based on two key features: 1) a bi-temporal DB specifically designed for handling historical and real-time data, 2) a GUI that aims to facilitate query formulation for business analysts. In this demonstration, we provide the key resources to let the visitors play with the Tornado functionalities to interact with predefined data.

One of the leading challenges for pervasive computing is to ease the application development to smoothly handle the surrounding environment. We consider the case where the environment produces heterogeneous and continuous data, e. g. temperature readings, car positions... We have defined a scenario for containers transportation tracking in a medical context involving the transportation of fragile biological matter in sensor-enhanced containers. This scenario has been simulated as a testbed and offers a very nice setting to measure the agility of data-centric application development.

Querying non-conventional data is recognized as a major issue in new environments and applications such as those occurring in pervasive computing. A key issue is the ability to query data, streams and services in a declarative way. Our overall objective is to make the development of pervasive applications easier through database principles. In this paper, through the notion of virtual attributes and binding patterns, we define a data-centric view of pervasive environments: the classical notion of database is extended to come up with a broader notion, defined as relational pervasive environment, integrating data, streams and active/passive services. Then, the so-called Serena algebra is proposed with operators to homogeneously handle data and services. Moreover, the notion of stream can also be smoothly integrated into this algebra. A prototype of Pervasive Environment Management System has been implemented on which first experiments have been conducted to validate our approach.

The discovery of frequent patterns is a famous problem in data mining. While plenty of algorithms have been proposed during the last decade, only a few contributions have tried to understand the influence of datasets on the algorithms behavior. Being able to explain why certain algorithms are likely to perform very well or very poorly on some datasets is still an open question. In this setting, we describe a thorough experimental study of datasets with respect to frequent item sets. We study the distribution of frequent item sets with respect to item sets size together with the distribution of three concise representations: frequent closed, frequent free and frequent essential item sets. For each of them, we also study the distribution of their positive and negative borders whenever possible. From this analysis, we exhibit a new characterization of datasets and some invariants allowing to better predict the behavior of well known algorithms. The main perspective of this work is to devise adaptive algorithms with respect to dataset characteristics.

Understanding data semantics from existing relational databases is important for several applications such as database maintenance and analysis, database re-engineering, data warehouse design or query optimization. We present a tool called DBA Companion, which can be a help to deal with the understanding of existing relational databases. The prototype integrates algorithms dedicated to database analysis. This task rests on data mining techniques, which allow to design efficient algorithms. Emphasis is put on algorithm efficiency to be able to address operational situations, and then discover FDs and INDs satisfied in a database instance. The tool follows a loosely coupled approach with the underlying DBMS for algorithm execution. For instance, IARs are generated in the DBMS, which allows the user to modify them and to reiterate the analysis process from these new relations.

In the relational model, inclusion dependencies (INDs) convey many information on data semantics. They generalize foreign keys, which are very popular constraints in practice. However, one seldom knows the set of satisfied INDs in a database. The IND discovery problem in existing databases can be formulated as a data-mining problem. We underline that the exploration of IND expressions from most general (smallest) INDs to most specific (largest) INDs does not succeed whenever large INDs have to be discovered. To cope with this problem, we introduce a new algorithm, called Zigzag, which combines the strength of levelwise algorithms (to find out some smallest INDs) with an optimistic criteria to jump more or less to largest INDs. Preliminary tests, on synthetic databases, are presented and commented on. It is worth noting that the main result is general enough to be applied to other data-mining problems, such as maximal frequent itemsets mining.

The authors propose a framework for a broad class of data mining algorithms for understanding existing databases: functional and approximate dependency inference, minimal key inference, example relation generation and normal form tests. We point out that the common data centric step of these algorithms is the discovery of agree sets. A set-oriented approach for discovering agree sets from database relations based on SQL queries is proposed. Experiments have been performed in order to compare the proposed approach with a data mining approach. We also present a novel way to extract approximate functional dependencies having minimal errors from agree sets.

The paper describes a method to cope with denormalized relational schemas in a database reverse engineering process. We propose two main steps to improve the understanding of data semantics. Firstly we extract inclusion dependencies by analyzing the equi join queries embedded in application programs and by querying the database extension. Secondly we show how to discover only functional dependencies which influence the way attributes should be restructured. The method is interactive since an expert user has to validate the presumptions on the elicited dependencies. Moreover, a restructuring phase leads to a relational schema in third normal form provided with key constraints and referential integrity constraints. Finally, we sketch how an entity relationship schema can be derived from such information.

This paper introduces a method of reverse engineering for operational relational databases. The conceptual schemas are derived using information extracted from data dictionaries, database extensions and application programs. Its main strength relies on the assumptions made on the a priori knowledge available about the database (only [Formula: see text] and/or [Formula: see text] constraints on attribute(s)) as well as the user competence. We argue that most of the knowledge needed to build a conceptual schema, if not described in the Data Description Language, is embedded in application programs under various forms. The method is therefore based on four main steps: firstly, application program analysis is performed and a set [Formula: see text] of equi-joins is obtained; secondly, a conceptual schema is derived from [Formula: see text], from the database extension and from the relational schema; thirdly, this conceptual schema is validated through an interactive dialogue with the expert user who is helped in this task by indications given by the method. Finally, a schema reorganization under user control is achieved to match the user requirements better. We introduce also how other kinds of queries can help the task of semantics discovery. Additionally, we precisely identify the phases when user interaction is needed. This method has been successfully validated on an operational database.