We present a new approach for independently computing compact sketches that can be used to approximate the inner product between pairs of high-dimensional vectors. Based on the Weighted MinHash algorithm, our approach admits strong accuracy guarantees that improve on the guarantees of popular linear sketching approaches for inner product estimation, such as CountSketch and Johnson-Lindenstrauss projection. Specifically, while our method exactly matches linear sketching for dense vectors, it yields significantly lower error for sparse vectors with limited overlap between non-zero entries. Such vectors arise in many applications involving sparse data, as well as in increasingly popular dataset search applications, where inner products are used to estimate data covariance, conditional means, and other quantities involving columns in unjoined tables. We complement our theoretical results by showing that our approach empirically outperforms existing linear sketches and unweighted hashing-based sketches for sparse vectors.

Dataset search is emerging as a critical capability in both research and industry: it has spurred many novel applications, ranging from the enrichment of analyses of real-world phenomena to the improvement of machine learning models. Recent research in this field has explored a new class of data-driven queries: queries consist of datasets and retrieve, from a large collection, related datasets. In this paper, we study a specific type of data-driven query that supports relational data augmentation through numerical data relationships: given an input query table, find the top-k tables that are both joinable with it and contain columns that are correlated with a column in the query. We propose a novel hashing scheme that allows the construction of a sketch-based index to support efficient correlated table search. We show that our proposed approach is effective and efficient, and achieves better trade-offs that significantly improve both the ranking accuracy and recall compared to the state-of-the-art solutions.

Conflict researchers face many challenges, including (1) how to model conflicts, (2) how to measure them, (3) how to manage their spatio-temporal character, and (4) how to handle a potential abundance of information and explanation. In this paper, we describe an ecosystem of tools designed for use by subject matter experts that addresses these challenges. Three case studies show workflows that are facilitated by this ecosystem.

The increasing availability of structured datasets, from Web tables and open-data portals to enterprise data, opens up opportunities to enrich analytics and improve machine learning models through relational data augmentation. In this paper, we introduce a new class of data augmentation queries: join-correlation queries. Given a column Q and a join column KQ from a query table TQ, retrieve tables TX in a dataset collection such that TX is joinable with TQ on KQ and there is a column C ∈ TX such that Q is correlated with C. A naïve approach to evaluate these queries, which first finds joinable tables and then explicitly joins and computes correlations between Q and all columns of the discovered tables, is prohibitively expensive. To efficiently support correlated column discovery, we 1) propose a sketching method that enables the construction of an index for a large number of tables and that provides accurate estimates for join-correlation queries, and 2) explore different scoring strategies that effectively rank the query results based on how well the columns are correlated with the query. We carry out a detailed experimental evaluation, using both synthetic and real data, which shows that our sketches attain high accuracy and the scoring strategies lead to high-quality rankings.

The large volumes of structured data currently available, from Web tables to open-data portals and enterprise data, open up new opportunities for progress in answering many important scientific, societal, and business questions. However, finding relevant data is difficult. While search engines have addressed this problem for Web documents, there are many new challenges involved in supporting the discovery of structured data. We demonstrate how the Auctus dataset search engine addresses some of these challenges. We describe the system architecture and how users can explore datasets through a rich set of queries. We also present case studies which show how Auctus supports data augmentation to improve machine learning models as well as to enrich analytics.

Increasingly, decisions are based on insights and conclusions derived from the results of data analysis. Thus, determining the validity of these results is of paramount importance. In this paper, we take a step towards helping users identify potential issues in spatio-temporal data and thus gain trust in the results they derived from these data. We focus on processes that are captured by relationships among datasets that serve as the data exhaust for different components of urban environments. In this scenario, debugging data involves two important challenges: the inherent complexity of spatio-temporal data, and the number of possible relationships. We propose a framework for profiling spatio-temporal relationships that automatically identifies data slices that present a significant deviation from what is expected, and thus, helps focus a user's attention on slices of the data that may have quality issues and/or that may affect the conclusions derived from the analysis' results. We describe the profiling methodology and how it derives relationships, identifies candidate deviations, assesses their statistical significance, and measures their magnitude. We also present a series of cases studies using real datasets from New York City which demonstrate the usefulness of spatio-temporal profiling to build trust on data analysis' results.

In this article we study the extent to which the interplay between recommended items affect recommendation effectiveness. We introduce and formalize the concept of co‐utility as the property that any pair of recommended items has of being useful to a user, and exploit it to improve collaborative filtering recommendations. We present different techniques to estimate co‐utility probabilities, all of them independent of content information, and compare them with each other. We use these probabilities, as well as normalized predicted ratings, in an instance of an ‐hard problem termed the Max‐Sum Dispersion Problem (MSDP). A solution to MSDP hence corresponds to a set of items for recommendation. We study one heuristic and one exact solution to MSDP and perform comparisons among them. We also contrast our solutions (the best heuristic to MSDP) to different baselines by comparing the ratings users give to different recommendations. We obtain expressive gains in the utility of recommendations and our solutions also recommend higher‐rated items to the majority of users. Finally, we show that our co‐utility solutions are scalable in practice and do not harm recommendations' diversity.