In this paper, we study the problem of numerical multi-table question answering (MTQA) over large-scale table collections (e.g., online data repositories). This task is essential in many analytical applications. Existing MTQA solutions, such as text-to-SQL or open-domain MTQA methods, are designed for databases and struggle when applied to large-scale table collections. The key limitations include: (1) Limited support for complex table relationships; (2) Ineffective retrieval of relevant tables at scale; (3) Inaccurate answer generation. To overcome these limitations, we propose DMRAL, a Decomposition-driven Multi-table Retrieval and Answering framework for MTQA over large-scale table collections, which consists of: (1) constructing a table relationship graph to capture complex relationships among tables; (2) Table-Aligned Question Decomposer and Coverage-Aware Retriever, which jointly enable the effective identification of relevant tables from large-scale corpora by enhancing the question decomposition quality and maximizing the question coverage of retrieved tables; and (3) Sub-question Guided Reasoner, which produces correct answers by progressively generating and refining the reasoning program based on sub-questions. Experiments on two MTQA datasets demonstrate that DMRAL significantly outperforms existing state-of-the-art MTQA methods, with an average improvement of 24% in table retrieval and 55% in answer accuracy.

Abstract Missing values in tabular data lakes can severely impact data analysis and diminish the performance in downstream applications. We highlight that a robust imputation strategy should properly take three aspects of variety into consideration: source of imputed value, the types of tables involved, and the data types of the missing value. Existing imputation methods rely on estimation-based approaches (using a model trained on data from the same table to estimate missing values) or search-based approaches (retrieving values from other tables). Unfortunately, none of these approaches effectively incorporate all three aspects of variety. To address this gap, we propose , a novel framework that uses a C ombination of E stimation-based and S earch-based methods for missing value I mputation in D ata lakes. contains three core modules: (1) the , which efficiently discovers candidate values from tables by exploiting the contextual information; (2) the , which introduces an influence function and a sampling-based exploration strategy to yield accurate estimated values; (3) the , which determines the most suitable method based on table-level and column-level statistics. Extensive experiments conducted on three data lakes demonstrate that effectively and efficiently addresses the missing value problem.

Line charts are a valuable tool for data analysis and exploration, distilling essential insights from a dataset. However, access to the underlying data used to create a line chart is rarely readily available. In this paper, we explore a novel dataset discovery problem, dataset discovery via line charts, focusing on the use of line charts as queries to discover datasets within a large data repository that are capable of generating similar line charts. To solve this problem, we propose a novel approach called Fine-grained Cross-modal Relevance Learning Model (FCM), which aims to estimate the relevance between a line chart and raw data from a candidate dataset. To achieve this goal, FCM first applies a visual element extractor to extract visual elements, i.e., lines and y-axis ticks, from a line chart. Then, two novel segment-level encoders are applied to learn representations for a line chart and a candidate dataset, preserving fine-grained information, followed by a cross-modal matcher that matchs the learned representations in a fine-grained manner. Furthermore, we extend FCM to support line chart query generated based on data aggregation. Last, we provide a benchmark tailored for this problem since no such dataset exists. Extensive evaluation on the new benchmark verifies the effectiveness of our proposed method. Specifically, our proposed approach surpasses the best baseline by 30.1% and 41.0% in terms of prec@50 and ndcg@50, respectively.

Abstract Table integration aims to create a comprehensive table by consolidating tuples containing relevant information. In this work, we investigate the challenge of integrating multiple tables from a data lake, focusing on three core tasks: (1) pairwise integrability judgment , which determines whether a tuple pair is integrable, accounting for any occurrences of semantic equivalence or typographical errors; (2) integrable set discovery , which identifies all integrable sets in a table based on pairwise integrability judgments established in the first task; (3) multi-tuple conflict resolution , which resolves conflicts between multiple tuples during integration. To this end, we train a binary classifier to address the task of pairwise integrability judgment. Given the scarcity of labeled data in data lakes, we propose a self-supervised adversarial contrastive learning algorithm to perform classification, which incorporates data augmentation methods and adversarial examples to autonomously generate new training data. Upon the output of pairwise integrability judgment, each integrable set can be considered as a community—a densely connected sub-graph where nodes and edges correspond to tuples in the table and their pairwise integrability, respectively—we proceed to investigate various community detection algorithms to address the integrable set discovery objective. Moving forward to tackle multi-tuple conflict resolution , we introduce an innovative in-context learning methodology. This approach capitalizes on the knowledge embedded within large language models to effectively resolve conflicts that arise when integrating multiple tuples. Notably, our method minimizes the need for annotated data, making it particularly suited for scenarios where labeled datasets are scarce. Since no suitable test collections are available for our tasks, we develop our own benchmarks using two real-world dataset repositories: Real and Join . We conduct extensive experiments on these benchmarks to validate the robustness and applicability of our methodologies in the context of integrating tables within data lakes.

Line charts are fundamental to data analysis and exploration, offering concise visual representations of trends. However, gaining access to the underlying data used to construct these charts is often challenging. In this paper, we describe DDLC (short for Dataset discovery via line charts), an automatic dataset discovery tool that is able to not only identify datasets (from a dataset repository) that are "relevant" to the information depicted from a line chart provided by the users, but also empower users to refine search results based on specific visual elements extracted from the line chart. Moreover, DDLC offers multiple avenues for users to validate search outcomes: 1) Providing explanations on how a similar line chart could be generated from the identified dataset; 2) enabling comparison of line charts generated from different datasets via different ways (e.g., the aggregation vs. non-aggregation operator); 3) facilitating fine-grained examination of the correspondence between the line chart and the identified dataset. By seamlessly combining dataset retrieval with visual refinement and validation mechanisms, DDLC offers a comprehensive solution for the data-driven exploration and analysis.

Visualization recommendation (VisRec) is to automatically generate the most relevant visualization for a table of interest to a user. In this paper, we present a novel machine learning-based VisRec method, VisFormer, which solves VisRec in three stages: 1) Table representation learning, which is to learn accurate column-level representations for a table. To achieve it, we resort to Transformer, a powerful language model that can learn accurate word embeddings by modeling context. Specifically, we propose a hierarchical Transformer-based architecture to learn expressive column representations by capturing two types of context, intra-column context and cross-column context; 2) Visual Relation Learning, which is to capture column relations. To achieve it, we regard each visualization as a relation tuple with a special relation, visual relation, between the columns. Then for each visual relation, we use a neural network to evaluate the corresponding visualizations; 3) Visual Preference Learning, which is to extract visual preference features that can affect users’ decision from a visualization. To achieve so, we use a Convolution Neural Network to extract such features and explore how to use them to refine the recommendation results. We conduct experiments to compare with three state-of-the-art ML-based methods on a large real-world dataset, Plotly community feed. The experimental results show that compared with the most competitive baseline, the relative improvements of VisFormer on Recall@1, Recall@2, and Recall@3 are 8.8%, 20.6%, and 21.0%, respectively.

In this paper, we propose a novel facility relocation problem where facilities (and their services) are portable, which is a combinatorial search problem with many practical applications. Given a set of users, a set of existing facilities, and a set of potential sites, we decide which of the existing facilities to relocate to potential sites, such that two factors are satisfied: (1) facility exposure: facilities after relocation have balanced exposure, namely serving equivalent numbers of users; (2) user convenience: it is convenient for users to access the nearest facility, which provides services with shorter travel distance. This problem is motivated by applications such as dynamically redistributing vaccine resources to align supply with demand for different vaccination centers, and relocating the bike sharing sites daily to improve the transportation efficiency. We first prove that this problem is NP-hard, and then we propose two algorithms: a non-learning best response algorithm () and a reinforcement learning algorithm (). In particular, the best response algorithm finds a Nash equilibrium to balance the facility-related and the user-related goals. To avoid being confined to only one Nash equilibrium, as found in the method, we also propose the reinforcement learning algorithm for long-term benefits, where each facility is an agent and we determine whether a facility needs to be relocated or not. To verify the effectiveness of our methods, we adopt multiple metrics to evaluate not only our objective, but also several other facility exposure equity and user convenience metrics to understand the benefits after facility relocation. Finally, comprehensive experiments using real-world datasets provide insights into the effectiveness of the two algorithms in practice.

In dynamic ridesharing, the platform serves as the mediator by tailoring the assignment result between workers and riders with a focus on a certain objective. Existing studies generally focus on either one or two stakeholders when modelling the problem while the wellbeing of the other parties may be ignored or even undermined. For example, purely maximizing the total revenue of the ridesharing platform may cause the loss of riders and in turn lead to a low served rate, because those expensive orders will be processed in priority. In this paper, we for the first time study how to incorporate the willingness of all stakeholders (i.e., the platform, workers and riders). Given a set of workers and a set of rider requests, we aim to return the matchable worker-rider pairs in order to minimize theregret. Specifically, two types of regret are defined: (i) theserved rate regret, which refers to the rate of unserved requests, catering for the reputation and profit of the platform and workers; (ii) therevenue regret, which considers the portion of revenue loss from unserved riders, catering for the focus of workers and riders in the trip schedule. We prove the NP-hardness of this problem. To tackle this problem, we first propose a dynamic programming insertion algorithm to improve the efficiency of inserting a rider request into a trip schedule of a worker. Furthermore, two kinds of heuristic algorithms are devised to match rider requests with workers effectively. Comprehensive experiments on two real-world datasets verify the effectiveness, efficiency and scalability of our solutions in dealing with different supply-demand relationships in practice.

Traffic bottlenecks are a set of road segments that have an unacceptable level of traffic caused by a poor balance between road capacity and traffic volume. A huge volume of trajectory data which captures realtime traffic conditions in road networks provides promising new opportunities to identify the traffic bottlenecks. In this paper, we define this problem as trajectory-driven traffic bottleneck identification : Given a road network R , a trajectory database T , find a representative set of seed edges of size K of traffic bottlenecks that influence the highest number of road segments not in the seed set. We show that this problem is NP-hard and propose a framework to find the traffic bottlenecks as follows. First, a traffic spread model is defined which represents changes in traffic volume for each road segment over time. Then, the traffic diffusion probability between two connected segments and the residual ratio of traffic volume for each segment can be computed using historical trajectory data. We then propose two different algorithmic approaches to solve the problem. The first one is a best-first algorithm BF , with an approximation ratio of 1-1/ e . To further accelerate the identification process in larger datasets, we also propose a sampling-based greedy algorithm SG . Finally, comprehensive experiments using three different datasets compare and contrast various solutions, and provide insights into important efficiency and effectiveness trade-offs among the respective methods.

In this paper, we propose and study a variant of the dynamic ridesharing problem with a specific focus on peak hours: Given a set of drivers and a set of rider requests, we aim to match drivers to each rider request by achieving two objectives: maximizing the served rate and minimizing the total additional distance, subject to a series of spatio-temporal constraints. Our problem can be distinguished from existing ridesharing solutions in three aspects: (1) Previous work did not fully explore the impact of peak travel periods where the number of rider requests is much greater than the number of available drivers. (2) Existing ridesharing solutions usually rely on single objective optimization techniques, such as minimizing the total travel cost (either distance or time). (3) When evaluating the overall system performance, the runtime spent on updating drivers' trip schedules as per newly coming rider requests should be incorporated, while it is unfortunately excluded by most existing solutions. In order to achieve our goal, we propose an underlying index structure on top of a partitioned road network, and compute the lower bounds of the shortest path distance between any two vertices. Using the proposed index together with a set of new pruning rules, we develop an efficient algorithm to dynamically include new riders directly into an existing trip schedule of a driver. In order to respond to new rider requests more effectively, we propose two algorithms that bilaterally match drivers with rider requests. Finally, we perform extensive experiments on a large-scale test collection to validate the effectiveness and efficiency of the proposed methods.

Existing spatial object recommendation algorithms generally treat objects identically when ranking them. However, spatial objects often cover different levels of spatial granularity and thereby are heterogeneous. For example, one user may prefer to be recommended a region (say Manhattan), while another user might prefer a venue (say a restaurant). Even for the same user, preferences can change at different stages of data exploration. In this paper, we study how to support top-k spatial object recommendations at varying levels of spatial granularity, enabling spatial objects at varying granularity, such as a city, suburb, or building, as a Point of Interest (POI). To solve this problem, we propose the use of a POI tree, which captures spatial containment relationships between POIs. We design a novel multi-task learning model called MPR (short for Multi-level POI Recommendation), where each task aims to return the top-k POIs at a certain spatial granularity level. Each task consists of two subtasks: (i) attribute-based representation learning; (ii) interaction-based representation learning. The first subtask learns the feature representations for both users and POIs, capturing attributes directly from their profiles. The second subtask incorporates user-POI interactions into the model. Additionally, MPR can provide insights into why certain recommendations are being made to a user based on three types of hints: user-aspect, POI-aspect, and interaction-aspect. We empirically validate our approach using two real-life datasets, and show promising performance improvements over several state-of-the-art methods.