Electronic health records (EHRs) store patient medical history in the structured data format, which facilitates automatic healthcare risk prediction, thereby improving personalized healthcare management and treatment. There are two main categories of methods for automatic healthcare risk prediction. The first models time-series information or relationships between visits for enhanced patient representations. However, given the high dimensionality nature of the EHR data, it often obtains compromise results due to the lack of training data. The second exploits external knowledge, e.g., knowledge graphs (KGs), to augment the training data, but less attention has been paid to distinguishing the importance of features and filtering out irrelevant external knowledge, leading to overwhelming noise and inefficiency. Additionally, the joint relationships between patient features were not emphasized, which are highlighted in clinical practice. In this paper, we propose an efficient Path Filtering with Causal Analysis (PFCA) approach for enhanced healthcare risk prediction to address these challenges. PFCA first extracts personalized knowledge graphs (PKGs) consisting of paths linking the patient's features to targets and then devises a fine-grained filtering method based on path messages to remove irrelevant paths for better efficiency. Then we develop an effective similarity-based method to model different features' joint interactions with targets to learn augmented representations for each feature. Furthermore, we design a causal analysis method that includes a novel causal intervention mechanism to mine and prioritize causal features for improved predictive performance. Finally, by exploiting the attention weights of paths in the PKGs, PFCA provides target-oriented interpretations, showing how patients' features lead to targets through significant paths. Experimental results on three public real-world datasets and four healthcare risk prediction tasks confirm PFCA's effectiveness in improving predictive performance compared to ten state-of-the-art baselines, demonstrate its efficiency of path filtering and interpretability.

Structured data prediction supports key applications in healthcare, finance, e-commerce, among others. As structured data becomes increasingly heterogeneous and complex, there is a growing need for scalable models that can generalize across diverse tables. Traditional machine learning and deep learning models for structured data often have task-specific architectures requiring computationally retraining for a new prediction task. Consequently, recent studies explore pre-training for structured data, which requires no additional training for a new task. While pre-training has achieved great success in NLP and CV, the research of pre-training for structured data remains preliminary because structured data has its unique characteristics, e.g., complicated correlations and dependencies between features. Existing studies adopt a fixed pre-trained model architecture and less attention has been paid to table-specific characteristics. This limits their ability to adapt to new prediction tasks. To address this challenge, we propose AixelNet, a pre-trained model for structured data prediction that supports table-aware adaptation. We first design a meta feature extraction module that summarizes table-level characteristics enabling model customization. Instead of relying on a single model, AixelNet adopts a multi-model framework with multiple base predictors to capture diverse feature interaction patterns across tables. Finally, these base predictor models are dynamically assigned to different tables for prediction based on the extracted table meta features via a designed hypernetwork, enabling flexible and table-aware model composition for different tables. To further improve generalization and efficiency, we design regularization methods to encourage balanced predictor model usage and diversified learned representations, as well as design a sparse update strategy for sparsely updating relevant predictor models during pre-training. Extensive experiments on 20 classification and 20 regression tasks on tables confirm AixelNet's effectiveness in improving predictive performance compared to eight state-of-the-art baselines and demonstrate its efficiency.

In the big data era, Table Question Answering (Table QA) has emerged as a crucial tool for extracting insights from structured data, especially in large table scenarios. There are two main categories of methods for Table QA: Executable Code-driven methods and Language Model based (LM-based) methods. Code-driven methods, e.g. Text-to-SQL based solutions, often struggle with incomplete or mismatching schema information. LM-based methods, include Pre-trained Language Models (PLMs) and Large Language Models (LLMs), also face challenges as PLMs have limited generalization, while LLMs suffer from performance degradation and increased token cost when applied to large tables. To address these challenges, we propose AixelAsk, a novel LLM-based framework designed for Large Table QA. Specifically, AixelAsk incorporates a three-module architecture consisting of Decomposition module, Retrieval module and Reasoning module. The Decomposition module constructs a directed acyclic graph (DAG)-based solution plan by decomposing the question into execution nodes with explicit dependencies, making a clear reasoning path to guide the LLM through a logical process. Inspired by the Retrieval-Augmented Generation, the Retrieval Module extracts key rows and columns from the large table, reducing input token size and focusing on critical information. The Reasoning Module performs step-by-step inferences over the retrieved sub-tables, guided by each execution node in the solution plan, to generate final answer. By tackling the challenges of LLM performance degradation with large inputs and complex questions, AixelAsk achieves superior performance in Large Table QA. Extensive experiments on various baselines across three datasets demonstrate the effectiveness and efficiency of our proposed AixelAsk framework. AixelAsk outperforms the state-of-the-art baseline by 4% - 8% in the exact match score, and at the same time reduces token usage by 86.4%, achieving both high accuracy and cost efficiency in the Large Table QA task.

Large Language Models (LLMs) have recently demonstrated exceptional performance in various real-world data management tasks through in-context learning (ICL), which involves structuring prompts with task descriptions and several demonstrations. However, most LLMs are not free and charge based on the number of input tokens. Specifically, for data management tasks, there may be massive related questions, leading to high inference cost due to redundant prompt content (i.e., overlapping demonstrations and repeated task descriptions). In this paper, we investigate the idea of batch prompting in leveraging LLMs for data management, which leads to cost-effective LLMs by grouping questions and demonstrations to perform inferences in batches. Current studies on batch prompting are preliminary and mostly based on heuristics, making it difficult to generalize to various types of tasks and adapt to different grouping strategies. To address these challenges, in this work we first formalize the batch prompting problem in general setting. Then, we study the hardness of this problem and propose efficient algorithms for adaptive grouping. Finally, we conduct comprehensive experiments on 14 datasets. Extensive experimental results demonstrate that our solution consistently outperforms the state-of-the-art baselines while consuming lower cost.

The generation of medical dialogue notes is essential in healthcare, providing a structured recapitalization of patient-provider interactions. Medical notes are rigorously organized into various sections, including Chief Complaint, History of Present Illness and more. Each section serves a specific purpose to record detailed medical content. Traditionally, this task is labor-intensive, requiring physicians to manually create notes, a process prone to errors. With advancements in AI, it is now feasible to automate the generation of medical notes. There are mainly two categories of methods for automatic medical note generation. Pre-trained language models (PLMs) struggle with unstructured outputs, limited datasets, and inadequate medical terminology. In-context learning (ICL) methods improve accuracy and reduce data requirements but still produce unstructured notes and require high time and cost. To tackle the above challenges, we propose a three-module framework, called CE-DEPT, for accurate, efficient and cost-effective medical note generation. Specifically, the Task Decomposition Module breaks down complete medical dialogues into section-specific dialogues to ensure relevance and accuracy. The Batch Combination Module groups these sections into batches based on disease similarity to reduce costs and improve efficiency. The Note Generation Module employs batch prompting with ICL to generate each section note, followed by combining them into a structured, comprehensive medical note. Experiments on benchmark datasets demonstrated the effectiveness of Task Decomposition and Batch Prompting. Our method, CE-DEPT outperforms the best method by 5% on the ROUGE-1 score, 3% on the Bertscore-F1, a cost-effectiveness improvement of 15%, and a reduction in time consumption of 25% at peak accuracy.

Data management is indispensable for informed decision-making in the big data era. In the meantime, Large Language Models (LLMs), equipped with billions of model parameters and trained on extensive data corpora, have recently achieved record-breaking results in various real-world applications, such as machine translation, content generation, information retrieval, etc. The emergent abilities of LLMs, e.g., in-context learning and advanced reasoning ability, have great potential to revolutionize data management. In this paper, we first present some promising categories of data management applications where LLMs can be adapted, including data generation, data transformation, data integration, and data exploration. We then discuss the corresponding challenges for such adaption. Finally, we envision potential solutions to these challenges.

Electronic Health Records, which contain abundant structured data information of the patients, can help clinicians and data scientists address complex medical issues, particularly medication recommendation. The recommendation of medications is crucial for accurate and timely prescriptions. It is a nuanced task that entails analyzing various sources of healthcare data. Traditional medication recommendation is performed manually, which is labor-intensive and error-prone. The development of Electronic Health Records enables automatic medication recommendation. There are mainly two categories of methods for automatic medication recommendation. The first category uses the patients' current visit information and the drug-drug interactions (DDI). For these methods, both the comprehensive patient's medical history and the significant medication-diagnosis knowledge are not exploited appropriately. The second category utilizes longitudinal patient data, but different history visits are incorporated indiscriminately. Furthermore, in clinical practice, the associations between historical medications and future prescriptions are highlighted. However, they are less emphasized in current methods. Nevertheless, this is less emphasized by current automatic medication recommendation methods. To tackle the above challenges, we propose a three-module Discharge Medication Recommendation Network, called DMRNet, for accurate discharge medication recommendations. Specifically, the Information Integration Module combines information from the current visit and significant external knowledge e.g., the Diagnosis-Medication Co-occurrence (DMC) relationship. The Medication Retention Module is specially designed to capture the associations between the historical medications and the recommended medications. The History Retrieval Module differentiates the significance of different historical visits and incorporates them based on different significance values. Experimental evaluations on benchmark datasets, i.e., MIMIC-III and MIMIC-IV, confirm DMRNet's superiority over state-of-the-art baseline methods in terms of Jaccard Similarity, F1-score, Precision and Recall.

The growing demand for advanced analytics beyond statistical aggregation calls for database systems that support effective model selection of deep neural networks (DNNs). However, existing model selection strategies are based on either training-based algorithms that deliver high-performing models at the expense of high computational cost, or training-free algorithms that enhance computational efficiency with reduced effectiveness. These strategies often disregard computational cost and response time Service-Level Objectives (SLOs), which are of concern to average or budget-conscious machine learning users. In addition, they lack a well-designed integration of the model selection algorithms with DBMSs, which hinders efficient in-database model selection. This paper presents TRAILS, a resource-efficient and SLO-aware in-database model selection system. To leverage the strengths of both training-free and training-based model selection, we first characterize nine state-of-the-art training-free model evaluation metrics and propose a more effective one named JacFlow, and then, restructure the conventional model selection procedure into two phases: filtering and refinement. A novel coordinator is also introduced to strike a balance between the high efficiency of train-free algorithms and the high effectiveness of training-based algorithms, ensuring high-performing model selection while adhering to target SLOs. Moreover, we incorporate the proposed algorithm into PostgreSQL to develop TRAILS, thereby both enhancing resource efficiency and reducing model selection latency. This integration establishes a foundation for declarative model definition and selection within DBMSs. Empirical results demonstrate that our TRAILS reduces model selection time and computational expenses considerably by up to 24.38x and 29.32x respectively compared to existing model selection systems.

Federated Learning (FL) emerges as a viable solution to facilitate data collaboration, enabling multiple clients to collaboratively train a machine learning (ML) model under the supervision of a central server while ensuring the confidentiality of their raw data. However, existing studies have unveiled two main risks: (i) the potential for the server to infer sensitive information from the client's uploaded updates (i.e., model gradients), compromising client input privacy, and (ii) the risk of malicious clients uploading malformed updates to poison the FL model, compromising input integrity. Recent works utilize secure aggregation with zero-knowledge proofs (ZKP) to guarantee input privacy and integrity in FL. Nevertheless, they suffer from extremely low efficiency and, thus, are impractical for real deployment. In this paper, we propose a novel and highly efficient approach RiseFL for secure and verifiable data collaboration, ensuring input privacy and integrity simultaneously. Firstly, we devise a probabilistic integrity check method that transforms strict checks into a hypothesis test problem, offering great optimization opportunities. Secondly, we introduce a hybrid commitment scheme to satisfy Byzantine robustness with improved performance. Thirdly, we present an optimized ZKP generation and verification technique that significantly reduces the ZKP cost based on probabilistic integrity checks. Furthermore, we theoretically prove the security guarantee of RiseFL and provide a cost analysis compared to state-of-the-art baselines. Extensive experiments on synthetic and real-world datasets suggest that our approach is effective and highly efficient in both client computation and communication. For instance, RiseFL is up to 28x, 53x, and 164x faster than baselines ACORN, RoFL, and EIFFeL for the client computation.

Feature selection is an effective technique for structured data analytics, aiming to eliminate redundant features and irrelevant features for downstream tasks (e.g., classification). With the deepening of data-driven decision-making applications in various industries, the demand for real-time structured data analysis is constantly increasing. At this time, high requirements are placed on the time cost of feature selection. However, existing feature selection methods may easily fall into the dilemma of efficiency and effectiveness when faced with this situation due to the huge feature space. In this paper, we study a novel fast feature selection scenario, which is to generalize the knowledge of feature selection from historical structured data analytics tasks (seen tasks) and then quickly apply it to the process of feature selection for future structured data analytics tasks (unseen tasks). We propose a novel Progress-Aware multi-task deep reinforcement learning method for Fast fEAture selecTion (PA-FEAT), which makes full use of various progress-related information generated during the knowledge generalization process to achieve efficiency and effectiveness simultaneously. Extensive results on eight real-world datasets show that PA-FEAT consistently outperforms eight baselines in terms of efficiency and effectiveness.

Heart is the most important organ of the human body, and Electrocardiogram (ECG) is an essential tool for clinical monitoring of heart health and detecting cardiovascular diseases. Automatic detection of ECG anomalies is of great significance and clinical value in healthcare. However, performing automatic anomaly detection for the ECG data is challenging because we not only need to accurately detect the anomalies but also need to provide clinically meaningful interpretation of the results. Existing works on automatic ECG anomaly detection either rely on hand-crafted designs of feature extraction algorithms which are typically too simple to deliver good performance, or deep learning for automatically extracting features, which is not interpretable.

In line with the increasing machine learning model inference accuracy, deep learning (DL) models have been increasingly applied to structured data for a wide spectrum of real-world applications, including product recommendations, online advertisement, healthcare analytics and risk analysis. However, unlike unstructured data, structured data is high-dimensional and sparse and therefore engenders a large number of parameters in DL, making DL models more prone to overfitting. To alleviate the overfitting problem, various regularization methods have been designed to constrain the model parameters as a means to control the model complexity. Unfortunately, these methods are often restricted to regularizing the parameter values directly without considering the intrinsic correlations and dependencies between attribute fields of structured data which is however key to effective structured data modeling. In this paper, we re-examine DL for structured data from a new perspective of attribute interactions. In particular, we seek to explicitly model and regularize the pairwise relationships between attribute fields of structured data, in a field-adaptive manner, via a proposed attentive and interpretable framework called ATT-Reg. Specifically, in this framework, a set of attentive weight matrices are introduced to each attribute field for modeling obviously different relationships with its neighboring attribute fields. Further, we derive from the Bayesian viewpoint a novel Attentive Regularization method for imposing adaptive regularization strengths on different pairs of attribute fields, based on the informativeness of their relationship, which is calculated using both data-driven information and functional dependency (FD) knowledge. Such adaptive regularization facilitates each attribute field to learn discriminative and diversified representations for more effective predictive analytics. We also develop a feature attribution method for supporting more interpretable predictions We validate the effectiveness of our ATT-Reg on six real-world datasets. Extensive experimental results show that ATT-Reg achieves significant improvement over state-of-the-art graph models, attentive models as well as regularization methods and supports an excellent degree of interpretation.

Federated learning (FL) enables multiple data owners to collaboratively train machine learning (ML) models without disclosing their raw data. In the vertical federated learning (VFL) setting, the collaborating parties have data from the same set of users but with disjoint attributes. After constructing the VFL models, the parties deploy the models in production systems to infer prediction requests. In practice, the prediction output itself may not be convincing for party users to make the decisions, especially in high-stakes applications. Model interpretability is therefore essential to provide meaningful insights and better comprehension on the prediction output. In this paper, we propose Falcon, a novel privacy-preserving and interpretable VFL system. First, Falcon supports VFL training and prediction with strong and efficient privacy protection for a wide range of ML models, including linear regression, logistic regression, and multi-layer perceptron. The protection is achieved by a hybrid strategy of threshold partially homomorphic encryption (PHE) and additive secret sharing scheme (SSS), ensuring no intermediate information disclosure. Second, Falcon facilitates understanding of VFL model predictions by a flexible and privacy-preserving interpretability framework, which enables the implementation of state-of-the-art interpretable methods in a decentralized setting. Third, Falcon supports efficient data parallelism of VFL tasks and optimizes the parallelism factors to reduce the overall execution time. Falcon is fully implemented, and on which, we conduct extensive experiments using six real-world and multiple synthetic datasets. The results demonstrate that Falcon achieves comparable accuracy to non-private algorithms and outperforms three secure baselines in terms of efficiency.

While recent developments of neural network (NN) models have led to a series of record-breaking achievements in many applications, the lack of sufficiently good datasets remains a problem for some applications. For such a problem, we can however exploit a large number of unstructured text corpora as an external knowledge to complement the training data, and most prevailing neural network solutions employ word embedding methods for such purposes. In this paper, we propose LDA-Reg, a novel knowledge driven regularization framework based on Latent Dirichlet Allocation (LDA) as an alternative to the word embedding methods to adaptively utilize abundant external knowledge and to interpret the NN model. For the joint learning of the parameters, we propose EM-SGD, an effective update method which incorporates Expectation Maximization (EM) and Stochastic Gradient Descent (SGD) to update parameters iteratively. Moreover, we also devise a lazy update and sparse update method for the high-dimensional inputs and sparse inputs respectively. We validate the effectiveness of our regularization framework through an extensive experimental study over real world and standard benchmark datasets. The results show that our proposed framework not only achieves significant improvement over state-of-the-art word embedding methods but also learns interpretable and significant topics for various tasks.

With the ever-increasing adoption of machine learning for data analytics, maintaining a machine learning pipeline is becoming more complex as both the datasets and trained models evolve with time. In a collaborative environment, the changes and updates due to pipeline evolution often cause cumbersome coordination and maintenance work, raising the costs and making it hard to use. Existing solutions, unfortunately, do not address the version evolution problem, especially in a collaborative environment where non-linear version control semantics are necessary to isolate operations made by different user roles. The lack of version control semantics also incurs unnecessary storage consumption and lowers efficiency due to data duplication and repeated data pre-processing, which are avoidable.In this paper, we identify two main challenges that arise during the deployment of machine learning pipelines, and address them with the design of versioning for an end-to-end analytics system MLCask. The system supports multiple user roles with the ability to perform Git-like branching and merging operations in the context of the machine learning pipelines. We define and accelerate the metric-driven merge operation by pruning the pipeline search tree using reusable history records and pipeline compatibility information. Further, we design and implement the prioritized pipeline search, which gives preference to the pipelines that probably yield better performance. The effectiveness of MLCask is evaluated through an extensive study over several real-world deployment cases. The performance evaluation shows that the proposed merge operation is up to 7.8x faster and saves up to 11.9x storage space than the baseline method that does not utilize history records.

Alphas are stock prediction models capturing trading signals in a stock market. A set of effective alphas can generate weakly correlated high returns to diversify the risk. Existing alphas can be categorized into two classes: Formulaic alphas are simple algebraic expressions of scalar features, and thus can generalize well and be mined into a weakly correlated set. Machine learning alphas are data-driven models over vector and matrix features. They are more predictive than formulaic alphas, but are too complex to mine into a weakly correlated set. In this paper, we introduce a new class of alphas to model scalar, vector, and matrix features which possess the strengths of these two existing classes. The new alphas predict returns with high accuracy and can be mined into a weakly correlated set. In addition, we propose a novel alpha mining framework based on AutoML, called AlphaEvolve, to generate the new alphas. To this end, we first propose operators for generating the new alphas and selectively injecting relational domain knowledge to model the relations between stocks. We then accelerate the alpha mining by proposing a pruning technique for redundant alphas. Experiments show that AlphaEvolve can evolve initial alphas into the new alphas with high returns and weak correlations.

Deep Learning and Machine Learning models have recently been shown to be effective in many real world applications. While these models achieve increasingly better predictive performance, their structures have also become much more complex. A common and difficult problem for complex models is overfitting. Regularization is used to penalize the complexity of the model in order to avoid overfitting. However, in most learning frameworks, regularization function is usually set with some hyper-parameters where the best setting is difficult to find. In this paper, we propose an adaptive regularization method, as part of a large end-to-end healthcare data analytics software stack, which effectively addresses the above difficulty. First, we propose a general adaptive regularization method based on Gaussian Mixture (GM) to learn the best regularization function according to the observed parameters. Second, we develop an effective update algorithm which integrates Expectation Maximization (EM) with Stochastic Gradient Descent (SGD). Third, we design a lazy update and sparse update algorithm to reduce the computational cost by 4x and 20x, respectively. The overall regularization framework is fast, adaptive, and easy-to-use. We validate the effectiveness of our regularization method through an extensive experimental study over 14 standard benchmark datasets and three kinds of deep learning/machine learning models. The results illustrate that our proposed adaptive regularization method achieves significant improvement over state-of-the-art regularization methods.

Deep Learning and Machine Learning models have recently been shown to be effective in many real world applications. While these models achieve increasingly better predictive performance, their structures have also become much more complex. A common and difficult problem for complex models is overfitting. Regularization is used to penalize the complexity of the model in order to avoid overfitting. However, in most learning frameworks, regularization function is usually set as some hyper parameters, and therefore the best setting is difficult to find. In this paper, we propose an adaptive regularization method, as part of a large end-to-end healthcare data analytics software stack, which effectively addresses the above difficulty. First, we propose a general adaptive regularization method based on Gaussian Mixture (GM) to learn the best regularization function according to the observed parameters. Second, we develop an effective update algorithm which integrates Expectation Maximization (EM) with Stochastic Gradient Descent (SGD). Third, we design a lazy update algorithm to reduce the computational cost by 4x. The overall regularization framework is fast, adaptive and easy-to-use. We validate the effectiveness of our regularization method through an extensive experimental study over 13 standard benchmark datasets and three kinds of deep learning/machine learning models. The results illustrate that our proposed adaptive regularization method achieves significant improvement over state-of-the-art regularization methods.

In this feasibility study, we demonstrate the use of a factor-graph-based probabilistic graphical model approach to process longitudinal data derived from a population's electronic health records (EHR). Processing of EHR allows for fore-casting patient-specific health complications and inference of population-level statistics on several epidemiological factors. As a case-study, we provide preliminary results and demonstrate feasibility of our approach by processing the EHR of a diabetic cohort in Singapore. Our model passes the feasibility test as we are able to forecast a series of health complications of a new patient based on the factor functions inferred from EHR of 100 diabetic patients spanning 10-years. This forecast gives both the caregivers and the patient a better view of the patient's health in the coming years and increases patient's motivation to stay healthy and conform to medication plan. Furthermore, our approach informs commonly occurring health complications in the population that warrant hospital readmissions, which helps a physician/clinician in decide when to intervene to avoid complications in order to improve the patient's quality of life and minimize the cost of care.