Large-scale diffusion models have demonstrated remarkable success across a variety of domains. These models not only exhibit exceptional performance in their primary tasks but also adapt well to downstream applications through the 'pre-train & fine-tune paradigm'. However, the potential misuse of diffusion models for generating unsafe content has raised significant concerns regarding their governance and regulation, necessitating robust unsafe output prevention strategies. Despite the urgent demand for mitigation techniques, a significant challenge persists: once a model is distributed for local deployment or fine-tuning, the model provider and third-party regulators relinquish control over the model's behavior.

The high sampling frequency of highway structural health monitoring systems brings a heavy burden on data storage. However, existing dynamic response identification approaches can guarantee either reduced data volume after identification or high accuracy of dynamic response identification. Motivated by this, we propose a real-time dynamic response identification method to filter meaningless data. Our method not only selects effective features from highway structural health monitoring data, but also designs a training data generation strategy for machine learning models within the dynamic response identification framework. Experimental results on real highway structural health monitoring data demonstrate that our proposed approach spends 0.4 ms to process the monitoring data generated in 1 s and saves around 91.63% storage space. Also, the recall value of our method achieves 0.91 on average.

Abstract In the process of responding to user applications on the highway structural health monitoring data sharing platforms, the objective is to decrease the network transmission costs and avoid a mass of redundant Input/Output operations between the storage server and local hard disks. Since none of existing job scheduling and structural health monitoring data analysis research has focused on this topic, we study the problem of cost-saving response scheduling for highway structural health monitoring data applications. To solve this problem, we develop a greedy response scheduler with (1+ $$\frac{1}{e-1})$$ -approximation ratio. Evaluation results demonstrate the effectiveness and efficiency of our proposed solution.

The Cloud-Edge-Device (CED) architecture has emerged as a new framework for real-time data processing in the Internet of Things (IoT) era. However, the edge and device face significant resource constraints that prevent them from storing or processing full datasets. Effective data partition across CED architectures is therefore critical for supporting real-time decision-making. However, existing static and coarse-grained dynamic methods fail to adapt to changing workloads and to meet real-time processing demands. To address this issue, we propose${\sf FinePar}$, a fine-grained dynamic data partitioning framework based on DRL, coupled with an efficient data allocation strategy.${\sf FinePar}$combines horizontal and vertical partition to optimize data partition across CED architectures to reduce data transfer volume and shorten execution time. We use DRL to adjust data partitioning strategies in real time based on task demands and resource states. To achieve end-to-end optimization, we design an efficient data allocation strategy. We verified the effectiveness of${\sf FinePar}$through extensive experiments. Experimental results show that${\sf FinePar}$can reduce the edge side latency by 80% under resource constraints and dynamically adapts to workload changes.

Research on learned cardinality estimation has made significant progress in recent years. However, existing methods still face distinct challenges that hinder their practical deployment in production environments. We define these challenges as the “Trilemma of Cardinality Estimation”, where learned cardinality estimation methods struggle to balance generality, accuracy, and updatability. To address these challenges, we introduce DistJoin, a join cardinality estimator based on efficient distribution prediction using multi-autoregressive models. Our contributions are threefold: (1) We propose a method to estimate join cardinality by leveraging the probability distributions of individual tables in a decoupled manner. (2) To meet the re quirements of efficiency for DistJoin, we develop Adaptive Neural Predicate Modulation (ANPM), a high-throughput distribution estimation model. (3) We demonstrate that an existing similar approach suffers from variance accumulation issues by formal variance analysis. To mitigate this problem, DistJoin employs a selectivity-based approach to infer join cardinality, effectively reducing variance. In summary, DistJoin not only represents the first data-driven method to support both equi and non-equi joins simultaneously but also demonstrates superior accuracy while enabling fast and flexible updates. The experimental results demonstrate that DistJoin achieves the highest accuracy, robustness to data updates, generality, and comparable update and inference speed relative to existing methods.

Storage scheduling is crucial for time series storage. However, designing an efficient hot and cold tiered storage scheduling strategy for time series across Cloud-Edge-Device (CED) architecture remains challenging. Although numerous research have studied hot and cold classification for relational data, these methods are not suitable for time series which has strong timeliness and complex access patterns. Therefore, in this paper, we present TempSched, a temperature-aware storage scheduler for time series across CED, which can identify hot and cold time series and predict data temperature efficiently to perform storage scheduling in advance. By employing Newton's law of cooling and the thermal radiation law, TempSched establishs a temperature model and encapsulates data temperature. It supports classifying hot and cold data and scheduling time series across CED. Subsequently, TempSched designs a workload prediction model and a frequent timestamp discovery algorithm to forecast access patterns and predict the future temperature. This can timely adjust to hot and cold storage. We validate TempSched on a public dataset, and the experimental results show that it can achieve about 94% hit rate for data access on the edge and device, which is 12% better than existing methods. It can help CED avoid storage overhead caused by storing the full data at all three sides, and greatly reduce data transfer overhead.

Data quality issues are prevalent in information systems, making data cleaning a complex and time-consuming task, particularly with large-scale datasets and the lack of standardized automated processes. Existing cleaning pipelines often lack automated schemes to guide the execution of cleaning algorithms and the sequence of error corrections, limiting their practicality in real-world big data applications. To address the growing demand for advanced cleaning tools driven by the complexity of data activities, we propose the UniClean framework for on-demand big data cleaning. UniClean employs a unified cleaning operation (Uniop) from multiple cleaners to optimize the data cleaning workflow. It integrates a cleaning parameter generation pipeline, a cleaning parameter selection pipeline, and a module for cleaning process preparation and optimization, covering the entire workflow from cleaner modeling and data preparation to optimal cleaning operation generation. UniClean provides an adaptive (data-driven cleaning workflow generation) and flexible (multi-signal extension system) solution to meet the urgent need for high-quality data in today's data-driven decision-making environments. We demonstrate how UniClean effectively addresses the challenges of big data cleaning across diverse information system landscapes.

The scale of data analysis tasks have increased, highlighting the critical importance of data quality. Data quality assessment and repair have become pivotal in data preparation. Despite the availability of numerous algorithms for data cleaning, these often focus on optimizing efficiency and minimizing labor costs, neglecting the explicit relationship between data quality management costs and benefits. This omission can lead to the failure of promising data analysis solutions. To address this, we propose CBAClean, a comprehensive system that integrates cost-benefit analysis into data cleaning. CBAClean aims to assist users in quantifying the costs of data quality management and providing optimal data cleaning solutions tailored to their needs. Key features include task-centered multi-perspective data quality assessment, a comprehensive data quality repair operator library, fine-grained human role division for effective cost control, and recommendation of optimal data cleaning solutions based on cost-benefit calculations. By incorporating cost-benefit analysis, CBAClean enhances the practical application of data quality management on real-world data governance platforms.

Denial constraints are vital in data quality management, but traditional mining algorithms struggle with time series data. To address this, we introduce a novel data quality rule, threshold Denial Constraints ($t$DCs), which enables predicate scaling in numerical contexts. We formalize the inference system for$t$DCs and demonstrate the monotonicity and abruptness of threshold predicates. To efficiently mine$t$DCs, we design the tDCDiscover algorithm, which leverages batch computation of differences and thresholds to significantly reduce the time required for acquiring homologous predicate evidence, achieving a 50% -66% decrease. Additionally, we introduce an evidence matrix to store evidence, lowering the complexity of evidence matching from$O(m)$to$O(1)$. We propose two pruning strategies: triviality pruning and prediction coverage pruning, to effectively decrease the search paths to one-fifth of their original number and eliminating at least 90% of unnecessary paths. We theoretically prove that tDCDiscover ensures minimal, valid, and complete results. Experimental results on eight real-world datasets demonstrate that, compared to the current state-of-the-art denial constraint mining techniques, tDCDiscover achieves more than double the efficiency when processing high-dimensional time series data. In downstream data cleaning tasks, tDCDiscover improves error detection precision by an average of 40% and repair accuracy by 18%, further offering advantages in time series data quality management.

Data wrangling (DW) is a fundamental step to prepare data for downstream mining tasks. Recent studies explore large language models (LLMs) to form a lightweight DW paradigm. Such studies typically require prompting an LLM with a DW task together with a few examples as task demonstrations (i.e., in-context learning). A problem yet to be explored is how to select the examples, to maximize task effectiveness given constraints on the size of the examples. To fill this gap, we introduce the constrained Shapley value (CSV), a tailored variant of the Shapley value with a constraint on the LLM prompt size, to guide example selection. We show that CSV has desirable properties in example importance estimation. Using CSV directly for LLM-based DW is still computationally intractable. We further propose activated contribution (ACSV) as an unbiased estimation for CSV and sample allocation algorithms with approximation guarantees. Empirical results show that, compared with DW examples manually selected by experts, CSV improves the effectiveness of LLMs for DW tasks including schema mapping, entity matching, error detection, and missing value imputation by 5.90% averagly in F1 score, demonstrating the general applicability of CSV for in-context learning example selection towards DW tasks.

Time series forecasting (TSF) is a prominent chal-lenge in data analytics, relevant to both scientific research and real-world industrial applications. The rapid increase in high-dimensional time series data has led researchers to develop numerous models capable of handling complex forecasting tasks across diverse scenarios. Nevertheless, selecting an appropriate model and optimizing its parameters-an issue known as the Combined Algorithm Selection and Hyperparameter optimization (CASH) problem-remains a significant challenge. It is worth investigating how to satisfy both accuracy and efficiency in selecting an optimal algorithm and its hyperparameter configu-ration for a given time series with minimal human intervention. Unfortunately, there is no such work in the field of TSF, which has been developed for more than a decade. Existing methods suffer low selection rate of optimal algorithms. Meanwhile, the TSF task is extremely algorithm-sensitive, and the prediction performance of different algorithms and hyperparameter settings on the same data varies greatly. In this paper, we propose a Proxy-Model-based meta-learning TSF-CASH approach named Auto- Tsf. In the offline training phase, Auto- Tsfextracts the historical experience based on the proxy models, which is used to guide the automatic algorithm selection in the online working phase. The historical experience extracted in the offline phase not only significantly reduces the time consumption for algorithm selection, but also the introduction of the proxy model enhances the optimal algorithm selection rate. Moreover, we propose an asynchronous parallel HPO method in the most time-consuming HPO stage, which further improves the efficiency of the whole TSF -CASH. The experimental results demonstrate that Auto- Tsfachieves SOTA in terms of performance and efficiency compared to existing CASH methods.

Entity Matching (EM) is a crucial task in data integration. Compared to deep learning-based EM solutions, tree-based machine learning models are more computationally effective and explainable, making them more applicable in real-world EM scenarios. However, Random Forest-based EM methods select features with a static feature engineering rule set for all attributes. Consequently, they suffer model retraining cost to select features, and can hardly customize to different EM tasks. To tackle this problem, we propose Description-Similarity Rules (DSR) for EM feature engineering. DSR introduces diverse attribute value distribution metrics and data-driven thresholds to traditional EM feature engineering rules. Unfortunately, both the DSR search space and its online model retraining costs are exponential. By pushing the model retraining to the offline stage, our DSR set mining algorithm is one order of magnitude faster than the baseline algorithms, taking only seconds for online selection. Empirically, DSR outperforms the feature engineering methods by 3.75 % on average F1 score, while reaching the state-of-the-art EM performance on several datasets.

The existing Click-Through Rate (CTR) algorithms have their own advantages and are sensitive to hyperparameters. Quickly obtaining a high-performance CTR model for the new task can bring good application effects. However, ordinary users fail to do so due to the lack of domain knowledge. In this paper, we remedy this deficiency by proposing AutoCTR, an efficient meta-learning based Combined Algorithm Selection and Hyperparameter Optimization (CASH) algorithm, to help non-expert users quickly find the best CTR model. In AutoCTR, we introduce the meta-learning technique to make full use of the meta-information w.r.t. CTR to guide for the new CTR task. Specifically, we utilize the meta-information to learn characteristics and representations of CTR algorithms with different settings. We use these meta experiences combined with few evaluation information on the target CTR dataset to efficiently exploring the huge CTR CASH search space for the new task. The CTR model representation method has significant influence on the quality of the learned meta experiences. To further enhance the experiences quality, we also design a Graph Neural Network (GNN) based embedding learning method. This method can link different CTR models through their components, and thus quickly learning higher-quality model representations. Extensive experimental results show that AutoCTR can quickly select suitable CTR models for different CTR tasks. Compared with the existing CASH algorithms, which ignore meta-information or rely on a huge amount of meta-information, AutoCTR is more reasonable and efficient.

Database vendors have made significant redesigns and developments to the relational database for providing cloud-hosted and cloud-native database services. Thus, the original knob-tuning experiences of DBAs are no longer applicable to the cloud database era. An interpretable estimation service is urgently needed to provide explicit guidance for database knob tuning. Unfortunately, less attention has been paid to estimating the performance of the knob configuration. To fill this gap, we propose EAST, a knob estimation system to provide interpretable & transferable knob estimation services for cloud databases. Firstly, we design an interpretable knob-embedding-based estimator to achieve the trusted and white-box knob estimation for researchers, practitioners, and even artificial intelligence knob tuners. Secondly, we design a two-stage transfer estimation approach by stacking ensemble learning to utilize historical experiences, improving time efficiency. Thirdly, EAST provides a user-friendly interface to support direct knob estimation and transfer knob estimation services. We have deployed our EAST11https://gitee.com/opengauss/openGauss-DBMind/tree/incubator/dbmind/components/knob_estimator to the DBMind component of OpenGauss and demonstrated the effectiveness of our system under the open-source benchmark TPCC.

The subsequence matching problem utilizing dynamic time warping as the similarity measurement has been recognized as a key operation in time series analysis for more than two decades. Existing index-free algorithms depend on DTW lower bounds to discard the unpromising candidate. However, these approaches typically cost O ( m ) time for each candidate, where m is the length of the query. Consequently, the overhead of computing the DTW lower bounds occupies a significant portion of the time in subsequence matching tasks. This paper proposes new algorithms capable of computing the DTW lower bounds in average O (log m ) time for each candidate, substantially alleviating this bottleneck of the subsequence matching problem. In addition, this paper designs novel DTW lower bounds according to the characteristics of the subsequence matching problem, which is more effective without introducing significant computational overhead. Based on the above improvements, an efficient subsequence matching algorithm called FSMDTW is designed. Experiments conducted on both real and synthetic datasets show that the proposed algorithm is about 2.6 times faster than SOTA on short and medium-length queries and up to one order of magnitude faster on longer queries.

Missing data prevalent in information systems impacts data diversity and fidelity, which systematically degrade clustering performance through biased similarity measures and unstable cluster boundaries. Current large-scale environments lack standardized imputation-clustering pipelines, as existing methods operate independently of downstream tasks without analyzing error propagation effects, leading to unreliable results. To address this, we propose TARImpute, a Task-Aware auto-Recommender system for missing value imputation for clustering. It owns three integrated features: Imputation Impact Profiler for quantitative evaluation of imputation-clustering interactions, Error Propagation Interpreter enabling explainable modeling of imputation error diffusion, and Adaptive Strategy Optimizer for dynamic selection of optimal imputation methods. TARImpute provides state-of-the-art imputation methods to evaluate their effects on clustering tasks. TARImpute also provides robust, interpretable solutions for low-quality data and shows extensibility to other analytical tasks.

Data cleaning is an essential technique to enhance data quality. Despite the proposal of various algorithms with different cleaning strategies, current automated cleaning technologies still fall short of practical requirements when dealing with large-scale data containing mixed errors. This paper presents UniClean to efficiently solve the mixed error cleaning problem with three key technical contributions. (1) A unified construction and extension method for cleaners, enabling cleaning methods to easily utilize various cleaners to perform cleaning tasks. (2) Three optimization strategies to achieve efficiency-oriented cleaning preparation. (3) A cleaning algorithm based on an optimized cleaning process to effectively clean mixed errors. UniClean achieves a time complexity of O (| D error | 4 · | Op | + |D| · | D error |), significantly enhancing scalability. Experiments on public and large-scale enterprise datasets demonstrate that UniClean achieves over 40% improvement across five metrics, compared to five state-of-the-art cleaning methods, and delivers more than 30% gains in F1 and REDR on complex datasets, while completing the cleaning process within hours even for millions of records.

Real-world datasets often suffer from multiple quality issues, hindering downstream model performance and increasing cleaning costs. To address this, we propose DemandClean, a reinforcement learning-based adaptive data cleaning framework that dynamically balances cleaning effectiveness and operational costs. DemandClean explicitly considers data authenticity (alignment with real-world facts), diversity (richness of feature values), and downstream models' noise tolerance. We categorize data errors as missing (reducing authenticity and diversity), semantic (affecting only authenticity), and syntactic (affecting authenticity but potentially increasing diversity). Based on these errors, DemandClean intelligently selects among Repair, Delete, or No actions, guided by error rates and model robustness. For interpretability, the framework visually distinguishes authenticity, diversity, and tolerance. Extensive experiments confirm that DemandClean achieves near-optimal accuracy at substantially reduced preprocessing costs. Specifically, it reduces repair actions by 80.0% and deletions by 80.7% compared to "Repair All" strategies, while maintaining or even exceeding their predictive performance, thus offering an interpretable, cost-effective, and scalable solution for practical applications.

Motif discovery is a critical operation for analyzing series data in many applications. Recent works demonstrate the importance of finding motifs with Dynamic Time Warping. However, existing algorithms spend most of their time in computing lower bounds of Dynamic Time Warping to filter out the unpromising candidates. Specifically, the time complexity for computing these lower bounds is$O(L)$for each pair of subsequences, where$L$is the length of the motif (subsequences). This paper proposes two new lower bounds, called$LB_{f}$and$LB_{M}$, both of them only cost amortized$O(1)$time for each pair of subsequences. On real datasets, the proposed lower bounds are at least one magnitude faster than the state-of-the-art lower bounds used in motif discovery while still keeping satisfying effectiveness. Based on these faster lower bounds, this paper designs an efficient motif discovery algorithm that significantly reduces the cost of lower bounds. The experiments conducted on real datasets show the proposed algorithm is 5.6 times faster than the state-of-the-art algorithms on average.

The dictionary-based approach is one of the most representative types of time series classification (TSC) algorithm due to its high accuracy, efficiency, and good interpretability. However, existing studies focus on the centralized scenario where data from multiple sources are gathered. Considering that in many practical applications, data owners are reluctant to share their data due to privacy concerns, we study an unexplored problem involving collaboratively building the dictionary-based model over the data owners without disclosing their private data (i.e., in the federated scenario). We propose FedDict, a novel dictionarybased TSC approach customized for the federated setting to benefit from the advantages of the centralized algorithms. To further improve the performance and practicality, we propose a novel federated optimization algorithm for training logistic regression classifiers using dictionary features. The algorithm does not rely on any secure broker and is more accurate and efficient than existing solutions without hyper-parameter tuning. We also propose two contract algorithms for federated dictionary building, such that the user can flexibly balance the running time and the TSC performance through a predefined time limit. Extensive experiments on a total of 117 highly heterogeneous datasets validate the effectiveness of our methods and the superiority over existing solutions.

The knob tuning aims to optimize database performance by searching for the most effective knob configuration under a certain workload. Existing works suffer from two significant problems. First, there exist multiple useless evaluations of knob tuning even with diverse searching methods because of the different sensitivities of knobs on a certain workload. Second, the single evaluation of knob configurations may bring overestimation or underestimation because of query performance uncertainty. To solve the above problems, we propose a query uncertainty-aware knob classifier, called${\sf KnobCF}$, to enhance knob tuning. Our method has three contributions: (1) We propose uncertainty-aware configuration estimation to improve the tuning process. (2) We design a few-shot uncertainty estimator that requires no extra data collection, ensuring high efficiency in practical tasks. (3) We provide a flexible framework that can be integrated into existing knob tuners and DBMSs without modification. Our experiments on four open-source benchmarks demonstrate that our method effectively reduces useless evaluations and improves the tuning results. Especially in TPCC, our method achieves competitive tuning results with only 60% to 70% time consumption compared to the full workload evaluations.

Intelligent devices often produce time series data that suffer from significant data quality issues. While the utilization of data dependency in error detection and data repair has been somewhat beneficial, it remains inadequate in accurately representing the data quality of time series datasets. In recognition of the obvious characteristics inherent in time series data, we introduce a novel data dependency, termed TSDD. It effectively captures the contextual relationships embedded within multivariate time series, thereby enhancing the semantic richness of data quality representations. We analyze the complexity of both implication and consistency problems for TSDD reasoning, and develop TSDD discovery algorithm TSDDISCOVER, which consists of functional structure discovery, allowable error bound determination, and validation of TSDD patterns. Experimental results on real-life datasets verify TSDDISCOVER efficiently discovers high-quality TSDD patterns. In comparing the performance of TSDD-based error detection with several leading data quality constraints, our findings reveal that the former achieves an average improvement of 12% in accuracy and 30% in the F1 score over other dependency-based detection methods.

Time series data generated by thousands of sensors are suffering data quality problems. Traditional constraint-based techniques have greatly contributed to data cleaning applications. However, cleaning methods that support expressive constraints on time series data remain insufficient. Given the notable characteristics of time series data, existing cleaning approaches are challenged to provide good repair solutions. To address the challenges, we propose a novel data cleaning method for time series which incorporates expressive constraints that support arithmetic operations between attributes and time context. In the violation detection phase, we introduce specialized violation degree quantification functions and design a violation cell discovery algorithm to identify errors hidden in time series data. In the data repairing phase, we formalize the cleaning task as a constrained optimization problem and develop a novel repair objective function that considers both modification costs and conformance degrees of constraints. We effectively reduce the repair search space through the evaluation of time-context constraints and propose a bidirectional repairing algorithm. We also provide theoretical analysis of the proposed repairing method. Experimental results on three real-world IoT datasets across five metrics demonstrate that our proposed method outperforms seven state-of-the-art cleaning techniques specialized for time series data. Specifically, we achieve a 60% improvement in repairing effectiveness and a 70% reduction in time costs with our designed cleaning strategy.

Assessing data quality through Functional Depen-dencies (FDs) is a crucial aspect of data governance. However, with the diverse range of data sources and the exponential growth in data volume, exact FDs can sometimes be impractical for real-world applications. In contrast, relaxed functional dependencies (RFDs), which allows for some flexibility in attribute value comparisons, demonstrates greater adaptability and flexibility for big data scenarios. To address the efficient discovery of RFDs, this paper proposes a novel mining method to supplement the current research gaps. By establishing a difference table for tuples, we transform the problem into a specialized minimal set covering problem. Additionally, we introduce two optimization strategies: reducing the time complexity of enumerating the left-hand side of the base RFDs to 0 (1) and decreasing the search complexity for feasible LHS attributes and threshold candidates from O(2m-l) to O(1.5m-1). We rigorously proof that our mining approach guarantees the identification of validity and minimal RFDs. Experiments on nine real-world datasets reveal that our method significantly improves efficiency compared to existing techniques. Furthermore, it uncovers more concise and higher-quality RFDs. Importantly, the RFDs extracted through our methodology exhibit better performance in downstream cleaning tasks.

In this era of exploding data volumes, more and more complex data analysis tasks are now accomplished by machine learning (ML) or deep learning (DL). Despite the powerful and flexible task processing capability, it also brings challenges such as how to reasonably design the optimal hyperparameters configuration and the huge consumption of time for a single validation. Existing Hyperrarameter Optimization (HPO) methods are gradually facing performance bottlenecks. However, in an era where progressively data-centric big data analytics methods are prevalent, such efficiency issues need to be urgently addressed in the design of intelligent DBMS. In this paper, we propose ShrinkHPO, an efficient and explainable-designed HPO approach with a major focus on ($a$) efficient hyperparameter configuration search strategy, (b) asynchronous executing intervention, and (c) XAI (eXplainable AI) design. ShrinkHPO employs a hyperparameter weight estimation strategy named Shrink-search together with an asynchronous execution design to improve HPO efficiency. To the best of our knowledge, ShrinkHPO is the first HPO method that introduces explainable analysis and verification to ensure the judgment of “high significance” hyperparameters. We also conduct a series of experiments on data analysis tasks in the database domain (classification, regression, time series classification, CASH [1], etc.), collecting HPO results on the usual ML or DL models in each task. Compared to SOTA asynchronous and sequential HPO baselines, ShrinkHPO achieves top performance on both accuracy (RMSE for regression tasks) and time cost, accelerating from 20% to a maximum of 2.2 x.

With the explosion of artificial intelligence in various fields, providing causal explanations for predictive models becomes urgent. In this study, we investigate causal counterfactual explanation generation and propose CUBE, a causal intervention-based counterfactual explanation method. This work models the counterfactual generation process as a causal intervention model for the first time and employs a causal director to integrate causal relationships. Furthermore, this work proposes a model-based framework to improve counterfactual generation efficiency. The experimental results validate that CUBE outperforms baselines in terms of both lower time costs and higher explanation quality.

Data management system configuration optimization has been the focus of database-related research. It can improve the adaptability of DBMS to various business scenarios by selecting the right parameter combination. However, this has been a daunting task because of the vast number of configuration “knobs” involved, and the information about the effects of each knob usually comes from valuable experience, which is very labor-intensive. While machine learning techniques show promise in certain database tasks, they encounter challenges when applied to configuration optimization. Traditional machine learning models excel in prediction tasks through data fitting, but they lack direct applicability in configuration tuning. Moreover, the lack of explainability in machine learning poses a significant obstacle in quantifying the impact of individual knobs on the database performance. Affected by the above factors, this paper proposes CFTune, a method that can accurately evaluate the performance of a DBMS under each configuration using experience and achieve automatic DBMS configuration tuning with counterfactual techniques. Based on various optimization requirements, the approach can offer configuration tuning advice with minimal modification knobs for DBMS that perform badly under specified workloads. Furthermore, to address model shifts resulting from training data, we offer additional strategies to enhance the robustness of the tuning advice. The experimental results demonstrate that the CFTune effectively optimizes the configuration and enhances the performance.

Computing strongly connected components (SCC) is a key operation for many applications on directed graphs. Specifically, a SCC of a directed graph$G$is one of its maximal subgraphs, in which any two nodes are reachable to each other.

Model compression methods can reduce model complexity on the premise of maintaining acceptable performance, and thus promote the application of deep neural networks under resource constrained environments. Despite their great success, the selection of suitable compression methods and design of details of the compression scheme are difficult, requiring lots of domain knowledge as support, which is not friendly to non-expert users. To make more users easily access to the model compression scheme that best meet their needs, in this paper, we propose AutoMC, an effective and efficient automatic tool for model compression. In order to improve the search efficiency and quality, in AutoMC, we build the domain knowledge on model compression to deeply understand the characteristics and advantages of each compression method under different settings. This method can provide AutoMC with the more reasonable guidance and thus reduce useless evaluation. In addition, we present a progressive search strategy to efficiently explore pareto optimal compression scheme according to the learned prior knowledge combined with the historical evaluation information. This strategy can help AutoMC selectively and gradually explore more valuable search space, and thus reduce the search difficulty and improve the search efficiency. Extensive experimental results show that AutoMC can provide users with better compression schemes within short time compared to the existing compression methods and AutoML algorithms, which demonstrates the effectiveness and significance of our proposed algorithm.

Query cost estimation is a classical task for database management. Recently, researchers have applied AI-driven methods to implement query cost estimation for achieving high accuracy. However, two defects of the feature design lead to poor time-accuracy efficiency in the query cost estimation task. On the one hand, existing works only encode the query plan and data statistics while ignoring some important variables, like storage structure, hardware, database knobs, etc. These variables also have a significant impact on the query cost. On the other hand, existing works suffer the heavy model training and model inference due to inefficient features, such as the index encoding of write-only workloads. To address the above two problems, we first propose an efficient feature engineering for query cost estimation, called QCFE, consisting of the feature snapshot and feature reduction algorithm. (1) We design a novel concept called feature snapshot to efficiently integrate the influences of the missing variables. (2) We propose a difference-propagation feature reduction method for query cost estimation to filter the ineffective features. Compared to state-of-the-art methods, QCFE demonstrates significant improvements in various aspects with well-known benchmarks. QCFE saves up to 50% time consumption for model training, resulting in more efficient and faster training processes. QCFE also optimizes the mean q-error by 19.8% in TPCH, leading to more precise query cost estimation. QCFE offers up to an impressive 8 times inference speedup in query inference throughput.

Learned cardinality estimation methods have achieved high precision compared to traditional methods. Among learned methods, query-driven approaches have faced the work-load drift problem for a long time. Although both data-driven and hybrid methods are proposed to avoid this problem, most of them suffer from high training and estimation costs, limited scalability, instability, and long-tail distribution problems on high-dimensional tables, which seriously affects the practical application of learned cardinality estimators. In this paper, we prove that most of these problems are directly caused by the widely used progressive sampling. We solve this problem by introducing predicate information into the autoregressive model and propose Duet, a stable, efficient, and scalable hybrid method to estimate cardinality directly without sampling or any non-differentiable process, which can not only reduce the inference complexity from$O(n)$to$O(1)$compared to Naru and UAE but also achieve higher accuracy on high cardinality and high-dimensional tables. Experimental results show that Duet can achieve all the design goals above and be much more practical. Besides, Duet even has a lower inference cost on CPU than that of most learned methods on GPU.

The Maximum Minimal Cut Problem (MMCP), a NP-hard combinatorial optimization (CO) problem, has not received much attention due to the demanding and challenging bi-connectivity constraint. Moreover, as a CO problem, it is also a daunting task for machine learning, especially without labeled instances. To deal with these problems, this work proposes an unsupervised learning framework combined with heuristics for MMCP that can provide valid and high-quality solutions. As far as we know, this is the first work that explores machine learning and heuristics to solve MMCP. The unsupervised solver is inspired by a relaxation-plus-rounding approach, the relaxed solution is parameterized by graph neural networks, and the cost and penalty of MMCP are explicitly written out, which can train the model end-to-end. A crucial observation is that each solution corresponds to at least one spanning tree. Based on this finding, a heuristic solver that implements tree transformations by adding vertices is utilized to repair and improve the solution quality of the unsupervised solver. Alternatively, the graph is simplified while guaranteeing solution consistency, which reduces the running time. We conduct extensive experiments to evaluate our framework and give a specific application. The results demonstrate the superiority of our method against two techniques designed.

The modern database has many precise and approximate counting requirements. Nevertheless, a solitary multidimensional index or cardinality estimator is insufficient to cater to the escalating demands across all counting scenarios. Such approaches are constrained either by query selectivity or by the compromise between query accuracy and efficiency. We propose CardIndex, a unified learned structure to solve the above problems. CardIndex serves as a versatile solution that not only functions as a multidimensional learned index for accurate counting but also doubles as an adaptive cardinality estimator, catering to varying counting scenarios with diverse requirements for precision and efficiency. Rigorous experimentation has showcased its superiority. Compared to the state-of-the-art (SOTA) autoregressive data-driven cardinality estimation baselines, our structure achieves training and updating times that are two orders of magnitude faster. Additionally, our CPU-based query estimation latency surpasses GPU-based baselines by two to three times. Notably, the estimation accuracy of low-selectivity queries is up to 314 times better than the current SOTA estimator. In terms of indexing tasks, the construction speed of our structure is two orders of magnitude faster than RSMI and 1.9 times faster than R-tree. Furthermore, it exhibits a point query processing speed that is 3%-17% times faster than RSMI and 1.07 to 2.75 times faster than R-tree and KDB-tree. Range queries under specific loads are 20% times faster than the SOTA indexes.

Data dependency mining plays a crucial role in understanding data relationships. To address the increasing complexities of real-world data, Approximate Functional Dependencies (AFDs) have been introduced, building upon traditional FD. However, existing AFD approaches use static relaxation coefficients, limiting their effectiveness in capturing dependencies in noisy data. We propose a dynamic AFD variant, DAFD, which incorporates attribute error rates. We establish a bijection between DAFD and FD, develop its inference system, and introduce DAFDiscover, an algorithm for mining dependencies directly on noisy data. DAFDiscover matches the time and space complexity of SOTA AFD mining methods while offering superior performance. We theoretically prove its correctness, provide a method for calculating DAFD probabilities (DAFD- prob ), and derive a lower bound for DAFD's validity on dirty data. Experimental results on multiple public datasets demonstrate the semantic superiority of DAFD and the effectiveness of DAFDiscover compared to existing SOTA AFD mining techniques.

The widespread existence of time series data in information systems poses significant challenges to data cleaning due to its quality issues, particularly the complex interdependencies among attributes and the persistence of errors. Existing semantic constraints, such as conditional regression rules and speed constraints, though helpful, remain insufficient for this task. This paper introduces two novel online cleaning methods: MTSClean and MTSClean- soft , designed to improve cleaning efficiency and robustness. By combining row and column constraints, we significantly accelerate the cleaning process, reducing the time complexity of the exact solution MTSClean from O (( NM ) 3.5 |Σ|) to O ( NM 3.5 |Σ|). Meanwhile, MTSClean- soft achieves O ( NM 2 ) and more precise repairs through optimized search for key cells and a novel repair cost function. Comparative experiments against nine benchmark methods highlight our approach's superiority in multiple metrics, completing cleaning tasks faster and performing better than state-of-the-art methods. This demonstrates the practicality and advantage of the proposed methods in cleaning multidimensional time series data.

Billions of data points are generated by devices equipped with thousands of sensors, leading to significant data quality issues in time series data. These errors not only complicate time series data management but also compromise the accuracy and reliability of analysis based on such data. Given the noteworthy characteristics of time series data, existing cleaning methods struggle to provide adequate repairs, and tools supporting expressive constraints for time series remain scarce. To address this, we develop Clean4TSDB, a specialized data cleaning system for time series databases. This system integrates three key modules: expressive data quality constraint discovery, violation detection, and multivariate time series repairing, forming a comprehensive "profiling-detection-repair" workflow. Technically, we introduce TSDD, a data quality constraint that effectively captures contextual relationships within multivariate time series, and implement an efficient algorithm for its automated mining. Leveraging both row- and column-based constraints, we propose an effective time series cleaning algorithm. From a system standpoint, Clean4TSDB is pre-configured for seamless integration with time series databases like Apache IoTDB. Using user-provided and algorithmically-mined constraints, it effectively identifies various error patterns and offers reliable cleaning solutions. Furthermore, we establish a comprehensive library of state-of-the-art time series repair algorithms to meet the diverse needs of different management scenarios.

Data series has been one of the significant data forms in various applications. It becomes imperative to devise a data series index that supports both approximate and exact similarity searches for large data series collections in high-dimensional metric spaces. The state-of-the-art works employ summarizations and indices to reduce the accesses to the data series. However, we discover two significant flaws that severely limit performance enhancement. Firstly, the state-of-the-art works often employ segment-based summarizations, whose lower bound distances decrease significantly when representing a data series collection, resulting in numerous invalid accesses. Secondly, the disk-based indices for the exact search mainly rely on tree-based indices, which results in low-quality approximate answers, consequently impacting the exact search. To address these problems, we propose a novel solution, Double Indices and Double Summarizations (DIDS). Besides segment-based summarizations, DIDS introduces reference-point-based summarizations to improve the pruning rate by the sorted-based representation strategy. Moreover, DIDS employs reference points and a cost model to cluster similar data series, and uses a graph-based approach to interconnect various regions, enhancing approximate search capabilities. We conduct experiments on extensive datasets, validating the superior search performance of DIDS.

Unsupervised (a.k.a. Self-supervised) representation learning (URL) has emerged as a new paradigm for time series analysis, because it has the ability to learn generalizable time series representation beneficial for many downstream tasks without using labels that are usually difficult to obtain. Considering that existing approaches have limitations in the design of the representation encoder and the learning objective, we have proposed Contrastive Shapelet Learning (CSL), the first URL method that learns the general-purpose shapelet-based representation through unsupervised contrastive learning, and shown its superior performance in several analysis tasks, such as time series classification, clustering, and anomaly detection. In this paper, we develop TimeCSL, an end-to-end system that makes full use of the general and interpretable shapelets learned by CSL to achieve explorable time series analysis in a unified pipeline. We introduce the system components and demonstrate how users interact with TimeCSL to solve different analysis tasks in the unified pipeline, and gain insight into their time series by exploring the learned shapelets and representation.

Federated learning (FL) is an attractive paradigm for privacy-aware distributed machine learning, which enables clients to collaboratively learn a global model without sharing clients’ data. Recently, many strategies have been proposed to improve the generality of the global model and thus improve FL effect. However, existing strategies either ignore the fairness among clients or sacrifice performance for fairness. They cannot ensure that the gap among clients is as small as possible without sacrificing federated performance. To address this issue, we propose ParetoFed , a new local information aggregation method dedicated to obtaining better federated performance with smaller gap among clients. Specifically, we propose to use multi-objective hyperparameter optimization (HPO) algorithm to gain global models that are both fair and effective. Then, we send Pareto Optimal global models to each client, allowing them to choose the most suitable one as the base to optimize their local model. ParetoFed not only make the global models more fair but also make the selection of local models more personalized, which can further improve the federated performance. Extensive experiments show that ParetoFed outperforms existing FL methods in terms of fairness, and even achieves better federated performance, which demonstrates the significance of our method.

Recent several years have witnessed the rapid explosion of artificial intelligence applied in various domains with the surpassing human-level performance. Despite the success, these models' underlying mechanisms remain a mystery, as their complicated representations make human understanding impossible. This mystery may cause discrimination and non-robustness in prediction. Making deep learning models more transparent and understandable is gaining popularity, but most of interpretation approaches provide spurious correlations leading to suboptimal, incorrect or even biased interpretations, which could be reduced by causal explanations. Motivated by this, we attempt to study the generation of causal explanations and proposeCUBE, a causal intervention-based counterfactual interpretation method. To ensure that the generation process of counterfactual explanation conforms to causality, we model the counterfactual generation process as a causal graph and construct a counterfactual generation model based on the causal intervention; to generate counterfactuals that adhere to the causality, we introduce a causal director to capture the causal relationships in the distribution and guide the generation of counterfactuals; to improve the efficiency of the counterfactual generation when facing a large number of explanation queries, we model it as a sample generation problem and propose an explainable framework based on adversarial generation. The experimental results validate thatCUBEoutperforms other approaches in terms of both lower time costs and higher explanation quality.

SequentialRecommendation (SR) System emerged recently as a powerful tool for suggesting users with the next item of interest. Despite their great success, the design of SR systems requires heavy manual work and domain knowledge. In this paper, we present$\mathbf {AutoSR}$, an effectiveAutomatedMachineLearning (AutoML) tool that enables automatic design of powerful SR systems based onGraphNeuralNetwork (GNN) andReinforcementLearning (RL). In$\mathbf {AutoSR}$, we summarize the design process of the SR systems and extract effective operations from the existing SR systems to construct our search space. Such an experience-based search space generates diverse SR systems by integrating effective operations of different systems, providing a basic condition for the implementation of AutoML. Besides, we propose a graph-based RL method to efficiently explore the SR search space, where operations have complex and diverse application conditions. Compared with the existing AutoML methods, which ignore potential relations among operations,$\mathbf {AutoSR}$can greatly avoid invalid SR system design and efficiently discover more powerful SR systems by analyzing the relation graph of various operations. Extensive experimental results show that$\mathbf {AutoSR}$can gain powerful SR systems, superior to the existing$\mathbf {AutoSR}$systems used for search space construction. Besides,$\mathbf {AutoSR}$is more efficient than the existing AutoML algorithms in SR system design, which demonstrate the superiority of$\mathbf {AutoSR}$.

With years of development, a significant number of Time Series Classification (TSC) algorithms have been proposed and applied to various fields such as scientific research and industry scenarios, including traditional statistical methods, machine learning methods, and recently deep learning models. However, choosing a suitable model along with good parameter values that perform well on a given task, which is also known as Combined Algorithm Selection and Hyperparameter optimization problem (CASH), is still challenging. How to automatically select the appropriate algorithm according to the task during analyzing is a topic worthy of further research. Nevertheless, for TSC, a field that has been developed for decades, there is no effective and efficient approach for automatic algorithm selection. To the best of our knowledge, the current approach is based on genetic search, which is very computationally intensive and time-consuming. Therefore, in this paper, we propose TSC-AutoML, a zero-configuration and meta-learning-based approach for the automatic Time Series Classification algorithm CASH (also known as TSC-CASH). TSC-AutoML extracts knowledge from historical tasks and performs automatic feature selection and knowledge filtering with a reinforcement learning policy. The experience extracted is filtered and transformed into metadata. The meta-learner trained on the metadata together with our proposed warm start strategy will select an optimal algorithm for tasks uploaded by users, and then our proposed Hyperparameter Optimization method based on the Fast Warm Start strategy searches for hyperparameter combinations of the selected algorithm and adjusts parameter configuration to achieve top performance. The entire process is pre-trained, automated for the new task, and parameter-free for the user to decide, making it easy for users with the little domain experience to get started easily. Experimental results illustrate that TSC-AutoML outperforms existing methods in terms of both time and accuracy of optimum algorithm selection.

Since knowledge graphs (KGs) describe and model the relationships between entities and concepts in the real world, reasoning on KGs often corresponds to the r eachability queries with l abel and s ubstructure c onstraints (LSCR queries). Specifically, for a search path p , LSCR queries not only require that the labels of the edges passed by p are in a label set, but also claim that a vertex in p could satisfy a substructure constraint.

In recent years, the problem of traffic congestion has become a hot topic. Accurate traffic flow prediction methods have received extensive attention from many researchers all over the world. Although many methods proposed at present have achieved good results in the field of traffic flow prediction, most of them only consider the static characteristic of traffic data, but do not consider the dynamic characteristic of traffic data. The factors that affect traffic flow prediction are changeable, and they will change over time. In response to this dynamic characteristic, the authors propose a model fusion mechanism based on transformer (TransFusion). The authors adopt two basic forecasting models (TCN and LSTM) as the underlying architectures. In view of the performance of different models on the traffic data at different times, the authors design a model fusion mechanism to assign dynamic weights to basic models at different times. Experiments on three datasets have proved that TransFusion has a significant improvement compared with basic models.

Recent studies have shown great promise in unsupervised representation learning (URL) for multivariate time series, because URL has the capability in learning generalizable representation for many downstream tasks without using inaccessible labels. However, existing approaches usually adopt the models originally designed for other domains (e.g., computer vision) to encode the time series data and rely on strong assumptions to design learning objectives, which limits their ability to perform well. To deal with these problems, we propose a novel URL framework for multivariate time series by learning time-series-specific shapelet-based representation through a popular contrasting learning paradigm. To the best of our knowledge, this is the first work that explores the shapelet-based embedding in the unsupervised general-purpose representation learning. A unified shapelet-based encoder and a novel learning objective with multi-grained contrasting and multi-scale alignment are particularly designed to achieve our goal, and a data augmentation library is employed to improve the generalization. We conduct extensive experiments using tens of real-world datasets to assess the representation quality on many downstream tasks, including classification, clustering, and anomaly detection. The results demonstrate the superiority of our method against not only URL competitors, but also techniques specially designed for downstream tasks. Our code has been made publicly available at https://github.com/real2fish/CSL.

In recent years, many spatial-temporal graph convolutional network (STGCN) models are proposed to deal with the spatial-temporal network data forecasting problem. These STGCN models have their own advantages, i.e., each of them puts forward many effective operations and achieves good prediction results in the real applications. If users can effectively utilize and combine these excellent operations integrating the advantages of existing models, then they may obtain more effective STGCN models thus create greater value using existing work. However, they fail to do so due to the lack of domain knowledge, and there is lack of automated system to help users to achieve this goal. In this article, we fill this gap and propose Auto-STGCN algorithm, which makes use of existing models to automatically explore high-performance STGCN model for specific scenarios. Specifically, we design Unified-STGCN framework, which summarizes the operations of existing architectures, and use parameters to control the usage and characteristic attributes of each operation, so as to realize the parameterized representation of the STGCN architecture and the reorganization and fusion of advantages. Then, we present Auto-STGCN, an optimization method based on reinforcement learning, to quickly search the parameter search space provided by Unified-STGCN, and generate optimal STGCN models automatically. Extensive experiments on real-world benchmark datasets show that our Auto-STGCN can find STGCN models superior to existing STGCN models used for search space construction, which demonstrates the effectiveness of our proposed method.

As the prosperity of unstructured data in networks, knowledge extraction tools have been designed for new knowledges from unstructured data streams. The generated RDF streams by knowledge extraction are always containing much errorous tuples causing inconsistency to knowledge graph engine.To enable the completeness of information from unstructured streams, dynamically repairing the violated RDF tuples is the best way to process. Observed this, we propose dynamic relation repair process to find and eliminate violations in errorous RDF stream. RDF data, arranged as graphs, leads to computation hardness when trying to find constraints and repairing metrics. In this paper, we consider graph repairing process with implicit graph constraints enabling RDF candidates validation and repairing through subgraph matching with the sample of localized subgraphs from graph engine with the same relation labels. We also propose approximated graph matching process through dynamic graph embedding for time efficiency. Cold start problem is also well analyzed to avoid inefficient repairing. Experimental results on real datasets demonstrate that our work can capture and repair violation in RDF streams dynamically and effectively.

To effectively manage increasing knowledge graphs in various domains, a hot research topic, knowledge graph storage management, has emerged. Existing methods are classified to relational stores and native graph stores. Relational stores are able to store large-scale knowledge graphs and convenient in updating knowledge, but the query performance weakens obviously when the selectivity of a knowledge graph query is large. Native graph stores are efficient in processing complex knowledge graph queries due to its index-free adjacent property, but they are inapplicable to manage a large-scale knowledge graph due to limited storage budgets or inflexible updating process. Motivated by this, we propose a dual-store structure which leverages a graph store to accelerate the complex query process in the relational store. However, it is challenging to determine what data to transfer from relational store to graph store at what time. To address this problem, we formulate it as a Markov Decision Process and derive a physical design tuner DOTIL based on reinforcement learning. With DOTIL, the dual-store structure is adaptive to dynamic changing workloads. Experimental results on real knowledge graphs demonstrate that our proposed dual-store structure improves query performance up to average 43.72% compared with the most commonly used relational stores.

In literature, many algorithms are proposed to find strongly connected components (SCC) for directed graphs. Specifically, a SCC of a directed graph$G$is one of its maximal subgraphs, in which any two nodes are reachable to each other. Existing in-memory algorithms are efficient, and can find all the SCCs of$G$in a linear time, with respect to the size of$G$. Nevertheless, as the sizes of graphs grow rapidly in real applications, current efforts have been focused on semi-external algorithms. Existing semi-external algorithms maintain an in-memory sketch$\mathcal {A}$of$G$, and gradually restructure$\mathcal {A}$with their in-memory processes (IMP) until all the SCCs can be computed based on$\mathcal {A}$. However, the I/O and CPU costs of existing algorithms are still high when$G$is relatively large. Thus, this paper proposes a new semi-external algorithmEP-SCCwith a novel IMPEP-Reductionfor finding all the SCCs of$G$efficiently. Extensive experiments are conducted on both synthetic and real graphs, in which WDC-2014 contains 1.7 billion nodes, and eu-2015 has over 91 billion edges. Experimental results confirm that EP-SCC significantly outperforms existing semi-external SCC algorithms.

Since knowledge graphs (KGs) describe and model the relationships between entities and concepts in the real world, reasoning on KGs often correspond to the reachability queries with label and substructure constraints (LSCR queries). Specifically, for a search path p, LSCR queries not only require that the labels of the edges passed by p are in a certain label set, but also claim that a vertex in p could satisfy a certain substructure constraint. They are much more complex than existing label-constraint reachability (LCR) queries. LSCR queries on KGs can be addressed by two natural ways (EA-1) an online search algorithm and (EA-2) a combined search strategy, to the best of our knowledge. This paper presents two optimized algorithms for EA-1 and EA-2, but the optimized algorithms are still inefficient, since their efficiencies are highly dominated by their search directions as analyzed in this paper. Motivated by that, this paper presents an efficient informed search strategy on KGs, named INSK, with a lightweight index, named local index. An extensive experimental evaluation, on both synthetic and real KGs, illustrates that our INSK can efficiently process LSCR queries on KGs.

GraphNeuralNetwork (GNN) has achieved great success in the field of graph data processing and analysis, but the design of GNN architecture is difficult and time-consuming. To reduce the development cost of GNNs, recently, someGNNNeuralArchitectureSearch (GNN NAS) techniques are presented for the automatic design of GNN architectures. These techniques bring great convenience to the use of GNN, but cannot be applied to the federated learning scenarios. They only consider the single-source graph dataset, while failing to deal with the distributed and private graph datasets, which limits their applications. To address this shortcoming, in this paper we propose FL-AGNNS, an efficient GNN NAS algorithm which enables distributed agents to cooperatively design powerful GNN models while keeping personal information on local devices. FL-AGNNS designs a novel federated evolutionary optimization strategy. This strategy can fully consider the GNN architectures favored by each client, thus recommend GNN architectures that perform well in multiple datasets. In additions, FL-AGNNS applies the GNN super-network, a weight sharing strategy, to speed up the evaluation of GNN models during the search phase. Extensive experimental results show that FL-AGNNS can recommend better GNN models in short time under the federated learning framework, surpassing the state-of-the-arts GNN models.

Recently, deep learning has been an area of intense research. However, as a kind of computing-intensive task, deep learning highly relies on the scale of GPU memory, which is usually prohibitive and scarce. Although some extensive works have been proposed for dynamic GPU memory management, they are hard to apply to systems with multiple dynamic workloads, such as in-database machine learning systems. In this paper, we demonstrated TENSILE, a method of managing GPU memory in tensor granularity to reduce the GPU memory peak, considering the multiple dynamic workloads. TENSILE tackled the cold-starting and across-iteration scheduling problem existing in previous works. We implemented TENSILE on a deep learning framework built by ourselves and evaluated its performance. The experiment results show that TENSILE can save more GPU memory with less extra overhead than prior works in single and multiple dynamic workloads scenarios.

Existing knowledge graph stores are classified to relational stores and native graph stores. Relational stores are able to store large-scale knowledge graphs and convenient in updating data, but the query performance weakens obviously when the selectivity of a knowledge graph query is large. Graph stores are efficient in processing complex knowledge graph queries, but they are inapplicable to manage a large-scale knowledge graph due to limited storage budgets or inflexible updating process. Motivated by this, we propose a dual-store structure which leverages a graph store to accelerate the complex query process in the relational store. However, it is challenging to determine that when we transfer which data partitions from relational store to graph store. To address this problem, we derive a physical design tuner DOTIL based on reinforcement learning. Experimental results demonstrate that the dual-store structure improves query performance up to average 50.11% compared with the most commonly used relational stores.

We demonstrate FedTSC, a novel federated learning (FL) system for interpretable time series classification (TSC). FedTSC is an FL-based TSC solution that makes a great balance among security, interpretability, accuracy, and efficiency. We achieve this by first extending the concept of FL to consider both stronger security and model interpretability. Then, we propose three novel TSC methods based on explainable features to deal with the challengeable FL problem. To build the model in the FL setting, we propose several security protocols that are well optimized by maximally reducing the bottlenecked communication complexity. We build the FedTSC system based on such a solution, and provide the user Sklearn-like Python APIs for practical utility. We show that the system is easy to use, and the novel TSC approach is superior.

Data quality plays a key role in big data management today. With the explosive growth of data from a variety of sources, the quality of data is faced with multiple problems. Motivated by this, we study the multiple data cleaning on incompleteness and inconsistency with currency reasoning and determination in this paper. We introduce a 4-step framework, named${\sf Imp3C}$, for errors detection and quality improvement in incomplete and inconsistent data without timestamps. We achieve an integrated currency determining method to compute the currency orders among tuples, according to currency constraints. Thus, the inconsistent data and missing values are repaired effectively considering the temporal impact. For both effectiveness and efficiency consideration, we carry out inconsistency repair ahead of incompleteness repair. A currency-related consistency distance metric is defined to measure the similarity between dirty tuples and clean ones more accurately. In addition, currency orders are treated as an important feature in the missing imputation training process. The solution algorithms are introduced in detail with case studies. A thorough experiment on three real-life datasets verifies our method${\sf Imp3C}$improves the performance of data repairing with multiple quality problems.${\sf Imp3C}$outperforms the existing advanced methods, especially in the datasets with complex currency orders.

To aggregate useful information among diversified sources, a hotspot research topic called truth discovery has emerged in recent years. Existing truth discovery methods attempt to infer the true attribute values for the entities by identifying and trusting reliable data sources. That is, the values provided by reliable sources are more likely to be the true values. However, all these methods neglect the relations among different entities, which play important roles in truth discovery task. When reliable data sources cannot provide sufficient information of entities, the true attribute values of these entities can still be inferred by propagating trustworthy information from related entities. Motivated by this, in this paper, we introduce theconstrained truth discoveryproblem. We incorporate denial constraints, a universally quantified first-order logic formalism which can express a large number of effective and widely existing relations among entities, into the process of truth discovery. We formulate it as a constrained optimization problem and analyze its hardness. To address the problem, we propose algorithms to partition the entities into disjoint groups, and generate arithmetic constraints for each disjoint group separately. Then, the true attribute values of the entities in each disjoint group are derived by minimizing the objective function under the corresponding arithmetic constraints. Experimental results on both real-world and synthetic datasets demonstrate that the proposed approach achieves good performance even with very few constraints and reliable sources.

With the growth of data from various sources, data quality is faced with multiple problems. In this paper, we study the multiple data cleaning on incompleteness and inconsistency with currency reasoning and determination. We introduce a 4-step method, named Imp3C, for error detection and repair in incomplete and inconsistent data without timestamps. We propose an integrated currency determining approach to compute currency order among tuples, thus, the dirty data can be repaired effectively considering the temporal impact. Experiments on three real-life datasets verify that Imp3C improves data repairing performance with multiple quality problems, especially in datasets with complex currency orders.

Concurrency control is a mechanism that database systems provide to allow multiple transactions to be executed at the same time while enforcing isolation. The concurrency control algorithm is key to performance of a database system. However, different concurrency control algorithms have different strengths and weaknesses, making each of them fits only for some types of workloads, while performs unsatisfactorily for others. As a result, the user will have to make assumptions about the workloads before choosing the concurrency control algorithm to achieve the best performance. To overcome this limitation, we propose a scheme, called transaction clustering, to decide the best isolation mechanism for any given pair of transactions automatically. Based on transaction clustering, we further develop the Clustering-based Concurrency Control algorithm, or C3 for short, which combines the pessimistic and the optimistic concurrency control algorithms to get the best of both worlds while mitigating their performance bottlenecks at the same time. Both theoretical and experimental studies show that, for high-conflict workloads, the performance of the C3 algorithm can be significantly better than the performance of both the pessimistic and the optimistic algorithms that C3 is based on.

Aggregating the information provided by multiple data sources, which is also known as information integration , plays an important role in data analytics. Since there often exists recording errors, intentional errors, conflicts and outdated data across different data sources, finding the true attribute values of each entity is a fundamental task of crucial importance [3] . The process to fulfill this task is called truth discovery , which has been extensively studied in the literature.

The increasing complexity of data analysis tasks makes it dependent on human expertise and challenging for non-experts. One of the major challenges faced in data analysis is the selection of the proper algorithm for given tasks and data sets. Motivated by this, we develop Assassin, aiming at helping users without enough expertise to automatically select optimal algorithms for classification tasks. By embedding meta-learning techniques and reinforced policy, our system can automatically extract experiences from previous tasks and train a meta-classifier to implement algorithm recommendations. Then we apply genetic search to explore hyperparameter configuration for the selected algorithm. We demonstrate Assassin with classification tasks from OpenML. The system chooses an appropriate algorithm and optimal hyperparameter configuration for them to achieve a high-level performance target. The Assassin has a user-friendly interface that allows users to customize the parameters during the search process.

The great amount and complex structure of graph data bring a big challenge to graph data management. However, traditional management approaches cannot tackle the challenge. Fortunately, reinforcement learning provides a new approach to solve this problem due to its automation and adaptivity in decision making. Motivated by this, we develop April, an automatic graph data management system, which performs storage structure selection, index selection, and query optimization based on reinforcement learning. The system selects storage structure, indices effectively and automatically, and optimizes the SPARQL queries efficiently. April also offers a friendly interface for users, which allows users to interact with the system in a customized mode. We demonstrate the effectiveness and efficiency of April with two graph data benchmarks.

Entity resolution (ER) is the problem of identifying and merging records that refer to the same real-world entity. In many scenarios, raw records are stored under heterogeneous environment. To leverage such records better, most existing work assume that schema matching and data exchange have been done to convert records under different schemas to those under a predefined schema. However, we observe that schema matching would lose information in some cases, which could be useful or even crucial to ER. To leverage sufficient information from heterogeneous sources, in this paper, we address several challenges of ER on heterogeneous records and show that none of existing similarity metrics or their transformations could be applied to find similar records under heterogeneous settings. Motivated by this, we propose a novel framework to iteratively find records which refer to the same entity as well as an index to generate candidates and accelerate similarity computation. Evaluations on real-world datasets show the effectiveness and efficiency of our methods.

In many fields, a mass of algorithms with completely different hyperparameters have been developed to address the same type of problems. Choosing the algorithm and hyperparameter setting correctly can promote the overall performance greatly, but users often fail to do so due to the absence of knowledge. How to help users to effectively and quickly select the suitable algorithm and hyperparameter settings for the given task instance is an important research topic nowadays, which is known as the CASH problem. In this paper, we design the Auto-Model approach, which makes full use of known information in the related research paper and introduces hyperparameter optimization techniques, to solve the CASH problem effectively. Auto-Model tremendously reduces the cost of algorithm implementations and hyperparameter configuration space, and thus capable of dealing with the CASH problem efficiently and easily. To demonstrate the benefit of Auto-Model, we compare it with classical Auto-Weka approach. The experimental results show that our proposed approach can provide superior results and achieves better performance in a short time.

Entity resolution (ER) is the problem of identifying and merging records that refer to the same real-world entity. In many scenarios, raw records are stored under heterogeneous environment. Specifically, the schemas of records may differ from each other. To leverage such records better, most existing work assume that schema matching and data exchange have been done to convert records under different schemas to those under a predefined schema. However, we observe that schema matching would lose information in some cases, which could be useful or even crucial to ER. To leverage sufficient information from heterogeneous sources, in this paper, we address several challenges of ER on heterogeneous records and show that none of existing similarity metrics or their transformations could be applied to find similar records under heterogeneous settings. Motivated by this, we design the similarity function and propose a novel framework to iteratively find records which refer to the same entity. Regarding efficiency, we build an index to generate candidates and accelerate similarity computation. Evaluations on real-world datasets show the effectiveness and efficiency of our methods.

Communities are prevalent in social networks, knowledge graphs, and biological networks. Recently, the topic of community search (CS), extracting a dense subgraph containing a query vertex q from a graph, has received great attention. However, existing CS solutions are designed for undirected graphs, and overlook directions of edges which potentially lose useful information carried on directions. In many applications (e.g., Twitter), users' relationships are often modeled as directed graphs (e.g., if a user a follows another user b, then there is an edge from a to b). In this paper, we study the problem of CS on directed graph. Given a vertex q of a graph G, we aim to find a densely connected subgraph containing q from G, in which vertices have strong interactions and high similarities, by using the minimum in/out-degrees metric. We first develop a baseline algorithm based on the concept of D-core. We further propose three index structures and corresponding query algorithms. Our experimental results on seven real graphs show that our solutions are very effective and efficient.

The current data explosion poses great challenges to approximate aggregation with high efficiency and accuracy. To address this problem, we propose a novel approach to calculate the aggregation answers with a high accuracy using only a small portion of the data. We introduce leverages to reflect individual differences in the data from a statistical perspective. Two kinds of estimators, the leverage-based estimator, and the sketch estimator (a "rough picture" of the aggregation answer), are in constraint relations and iteratively improved according to the actual conditions until their difference is below a threshold. Due to the iteration mechanism and the leverages, our approach achieves a high accuracy. Moreover, some features, such as not requiring recording the sampled data and easy to extend to various execution modes, such as the online mode, make our approach well suited to deal with big data. Experiments show that our approach has an extraordinary performance, and when compared with the uniform sampling, our approach can achieve high-quality answers with only 1/3 sample size.

In this paper, we deal with the problem of rule-based entity resolution on imprecise temporal data. We use record matching dependencies and data currency constraints to derive temporal records' information and trend of their attributes' evolvement with elapsing of time. We firstly block records into smaller blocks, and then by exploring data currency constraints. We propose a temporal clustering approach with two steps, i.e., the skeleton clustering and the banding clustering. Experiments show that our method achieves both high accuracy and efficiency with hidden temporal information on datasets without imprecise timestamps.

A reasonable and effective community scoring function is of great significance since it can measure the community quality of groups we found more properly and help us discover more valuable communities. In this paper, we propose a new community scoring function, ECOQUG. Different from the existing community scoring functions, ECOQUG is designed based on the experimental study and theoretical analysis of groups with different community qualities. ECOQUG is more convincing. In addition, we design a series of experiments to examine the effectiveness and accuracy of ECOQUG and 13 other classic community scoring functions comprehensively. The extensive experimental results show that ECOQUG is effective and better than other community scoring functions.

The great amount of time series generated by machines has enormous value in intelligent industry. Knowledge can be discovered from high-quality time series, and used for production optimization and anomaly detection in industry. However, the original sensors data always contain many errors. This requires a sophisticated cleaning strategy and a well-designed system for industrial data cleaning. Motivated by this, we introduce Cleanits, a system for industrial time series cleaning. It implements an integrated cleaning strategy for detecting and repairing three kinds of errors in industrial time series. We develop reliable data cleaning algorithms, considering features of both industrial time series and domain knowledge. We demonstrate Cleanits with two real datasets from power plants. The system detects and repairs multiple dirty data precisely, and improves the quality of industrial time series effectively. Cleanits has a friendly interface for users, and result visualization along with logs are available during each cleaning process.

Communities are prevalent in social networks, knowledge graphs, and biological networks. Recently, the topic of community search (CS), extracting a dense subgraph containing a query vertex q from a graph, has received great attention. However, existing CS solutions are designed for undirected graphs, and overlook directions of edges which potentially lose useful information carried on directions. In many applications (e.g., Twitter), users' relationships are often modeled as directed graphs (e.g., if a user a follows another user b, then there is an edge from a to b). In this paper, we study the problem of CS on directed graph. Given a vertex q of a graph G, we aim to find a densely connected subgraph containing q from G, in which vertices have strong interactions and high similarities, by using the minimum in/out-degrees metric. We first develop a baseline algorithm based on the concept of D-core. We further propose three index structures and corresponding query algorithms. Our experimental results on seven real graphs show that our solutions are very effective and efficient. For example, on a graph with over 1 billion of edges, we only need around 40mins to index it and 1~2sec to answer a query.

In this paper, we deal with the problem of rule-based entity resolution on imprecise temporal data. Entity resolution (ER) is widely explored in research community, but the problem on temporal data, especially without available timestamps, has not been studied well yet. Because of the elapsing of time, records referring to the same entity observed in different time periods may be different. Besides traditional similarity-based ER approaches, by carefully exploring several data quality rules, e.g., matching dependency and data currency, much information can be obtained to facilitate to cope with this problem. In this paper, we use such rules to derive temporal records' information of time order and trend of their attributes' evolvement with elapsing of time. Specifically, we first block records into smaller blocks, and then by exploring data currency constraints, we propose a temporal clustering approach with two steps, i.e., the skeleton clustering and the banding clustering. Experimental results on both real and synthetic data show that our entity resolution method can achieve both high accuracy and efficiency on datasets with hidden temporal information.

We describe CleanCloud, a system for cleaning big data based on Map-Reduce paradigm in cloud. Using Map-Reduce paradigm, the system detects and repairs various data quality problems in big data. We demonstrate the following features of CleanCloud: (a) the support for cleaning multiple data quality problems in big data; (b) a visual tool for watching the status of big data cleaning process and tuning the parameters for data cleaning; (c) the friendly interface for data input and setting as well as cleaned data collection for big data. CleanCloud is a promising system that provides scalable and effect data cleaning mechanism for big data in either files or databases.

A fundamental problem of time series is k nearest neighbor (k-NN) query processing. However, existing methods are not fast enough for large dataset. In this paper, we propose a novel approach, STS3, to process k-NN queries by transforming time series to sets and measure the similarity under Jaccard metric. Our approach is more accurate than Dynamic Time Warping(DTW) in our suitable scenarios and it is faster than most of the existing methods, due to the efficient similarity search for sets. Besides, we also developed an index, a pruning and an approximation technique to improve the k-NN query procedure. As shown in the experimental results, all of them could accelerate the query processing effectively.

Since regular expressions are often used to detect errors in sequences such as strings or date, it is natural to use them for data repair. Motivated by this, we propose a data repair method based on regular expression to make the input sequence data obey the given regular expression with minimal revision cost. The proposed method contains two steps, sequence repair and token value repair. For sequence repair, we propose the Regular-expression-based Structural Repair (RSR in short) algorithm. RSR algorithm is a dynamic programming algorithm that utilizes Nondeterministic Finite Automata (NFA) to calculate the edit distance between a prefix of the input string and a partial pattern regular expression with time complexity of O ( nm 2 ) and space complexity of O ( mn ) where m is the edge number of NFA and n is the input string length. We also develop an optimization strategy to achieve higher performance for long strings. For token value repair, we combine the edit-distance-based method and associate rules by a unified argument for the selection of the proper method. Experimental results on both real and synthetic data show that the proposed method could repair the data effectively and efficiently.

In this demo, we present Cleanix, a prototype system for cleaning relational Big Data. Cleanix takes data integrated from multiple data sources and cleans them on a shared-nothing machine cluster. The backend system is built on-top-of an extensible and flexible data-parallel substrate - the Hyracks framework. Cleanix supports various data cleaning tasks such as abnormal value detection and correction, incomplete data filling, de-duplication, and conflict resolution. We demonstrate that Cleanix is a practical tool that supports effective and efficient data cleaning at the large scale.

The retrieval results of online products information in e-commerce web sites are often difficult for users to use because of different descriptions for the same product. This paper proposes ProductSeeker, a product retrieval system organizing results according to their referring real-world entities for the conveniences of users. In the demonstration, we will present our system providing friendly interface to retrieve fresh product information and refining results according to feedback.

Entity resolution (ER) is to find the data objects referring to the same real-world entity. When ER is performed on relations, the crucial operator is record matching, which is to judge whether two tuples refer to the same real-world entity. Record matching is a longstanding issue. However, with massive and complex data in applications, current methods cannot satisfy the requirements. A Sequence-rule-based record matching (SeReMatching) is presented with the consideration of both which attributes should be used and their importance in record matching. We have changed the Bloom filter and therefore the checking speed is greatly increased. The best performance of the algorithm makes the complexity of entity resolution O (n). And extensive experiments were performed to evaluate our methods.

The ability to handle large scale graph data is crucial to an increasing number of applications. Much work has been dedicated to supporting basic graph operations such as subgraph matching, reachability, regular expression matching, etc. In many cases, graph indices are employed to speed up query processing. Typically, most indices require either super-linear indexing time or super-linear indexing space. Unfortunately, for very large graphs, super-linear approaches are almost always infeasible. In this paper, we study the problem of subgraph matching on billion-node graphs. We present a novel algorithm that supports efficient subgraph matching for graphs deployed on a distributed memory store. Instead of relying on super-linear indices, we use efficient graph exploration and massive parallel computing for query processing. Our experimental results demonstrate the feasibility of performing subgraph matching on web-scale graph data.

In this paper, we consider the entity resolution(ER) problem, which is to identify objects referring to the same real-world entity. Prior work of ER involves expensive similarity comparison and clustering approaches. Additionally, the quality of entity resolution may be low due to insufficient information. To address these problems, by adopting context information of data objects, we present a novel framework of entity resolution, context-based entity description (CED), to make context information help entity resolution. In our framework, each entity is described by a set of CEDs. During entity resolution, objects are only compared with CEDs to determine its corresponding entity. Additionally, we propose efficient algorithms for CED discovery and CED-based entity resolution. We experimentally evaluated our CED-based ER algorithm on the real DBLP datasets, and the experimental results show that our algorithm can achieve both high precision and recall as well as outperform existing methods.

In a variety of emerging applications one needs to decide whether a graph G matches another G p , i.e. , whether G has a topological structure similar to that of G p . The traditional notions of graph homomorphism and isomorphism often fall short of capturing the structural similarity in these applications. This paper studies revisions of these notions, providing a full treatment from complexity to algorithms. (1) We propose p-homomorphism (p -hom) and 1-1 p -hom, which extend graph homomorphism and subgraph isomorphism, respectively, by mapping edges from one graph to paths in another, and by measuring the similarity of nodes . (2) We introduce metrics to measure graph similarity, and several optimization problems for p -hom and 1-1 p -hom. (3) We show that the decision problems for p -hom and 1-1 p -hom are NP-complete even for DAGs, and that the optimization problems are approximation-hard. (4) Nevertheless, we provide approximation algorithms with provable guarantees on match quality. We experimentally verify the effectiveness of the revised notions and the efficiency of our algorithms in Web site matching, using real-life and synthetic data.

When XML documents are modeled as graphs, many research issues arise. In particular, there are many new challenges in query processing on graph-structured XML documents because traditional query processing techniques for tree-structured XML documents cannot be directly applied. This paper studies the problem of structural queries on graph-structured XML documents. A hash-based structural join algorithm, HGJoin, is first proposed to handle reachability queries on graph-structured XML documents. Then, it is extended to the algorithms to process structural queries in form of bipartite graphs. Finally, based on these algorithms, a strategy to process subgraph queries in form of general DAGs is proposed. Analysis and experiments show that all the algorithms have high performance. It is notable that all the algorithms above can be slightly modified to process structural queries in form of general graphs.