Learning curve extrapolation predicts neural network performance from early training epochs and has been applied to accelerate AutoML, facilitating hyperparameter tuning and neural architecture search. However, existing methods typically model the evolution of learning curves in isolation, neglecting the impact of neural network (NN) architectures, which influence the loss landscape and learning trajectories. In this work, we explore whether incorporating neural network architecture improves learning curve modeling and how to effectively integrate this architectural information. Motivated by the dynamical system view of optimization, we propose a novel architecture-aware neural differential equation model to forecast learning curves continuously. We empirically demonstrate its ability to capture the general trend of fluctuating learning curves while quantifying uncertainty through variational parameters. Our model outperforms current state-of-the-art learning curve extrapolation methods and pure time-series modeling approaches for both MLP and CNN-based learning curves. Additionally, we explore the applicability of our method in Neural Architecture Search scenarios, such as training configuration ranking.

Graphical event models (GEMs) are representations of temporal point process dynamics between different event types. Many real-world applications however involve limited event stream data, making it challenging to learn GEMs from data alone. In this paper, we introduce approaches that can work together in a score-based learning paradigm, to augment data with potentially different types of background knowledge. We propose novel scores for learning an important parametric class of GEMs; in particular, we propose a Bayesian score for leveraging prior information as well as a more practical simplification that involves fewer parameters, analogous to Bayesian networks. We also introduce a framework for incorporating easily assessed qualitative background knowledge from domain experts, in the form of statements such as `event X depends on event Y' or `event Y makes event X more likely'. The proposed framework has Bayesian interpretations and can be deployed by any score-based learner. Through an extensive empirical investigation, we demonstrate the practical benefits of background knowledge augmentation while learning GEMs for applications in the low-data regime.

Streams of irregularly occurring events are commonly modeled as a marked temporal point process. Many real-world datasets such as e-commerce transactions and electronic health records often involve events where multiple event types co-occur, e.g. multiple items purchased or multiple diseases diagnosed simultaneously. In this paper, we tackle multi-label prediction in such a problem setting, and propose a novel Transformer-based Conditional Mixture of Bernoulli Network (TCMBN) that leverages neural density estimation to capture complex temporal dependence as well as probabilistic dependence between concurrent event types. We also propose potentially incorporating domain knowledge in the objective by regularizing the predicted probability. To represent probabilistic dependence of concurrent event types graphically, we design a two-step approach that first learns the mixture of Bernoulli network and then solves a least-squares semi-definite constrained program to numerically approximate the sparse precision matrix from a learned covariance matrix. This approach proves to be effective for event prediction while also providing an interpretable and possibly non-stationary structure for insights into event co-occurrence. We demonstrate the superior performance of our approach compared to existing baselines on multiple synthetic and real benchmarks.

Discovering knowledge about which types of events influence others, using datasets of event sequences without time stamps, has several practical applications. While neural sequence models are able to capture complex and potentially long-range historical dependencies, they often lack the interpretability of simpler models for event sequence dynamics. We provide a novel neural framework in such a setting - a probabilistic attention-to-influence neural model - which not only captures complex instance-wise interactions between events but also learns influencers for each event type of interest. Given event sequence data and a prior distribution on type-wise influence, we efficiently learn an approximate posterior for type-wise influence by an attention-to-influence transformation using variational inference. Our method subsequently models the conditional likelihood of sequences by sampling the above posterior to focus attention on influencing event types. We motivate our general framework and show improved performance in experiments compared to existing baselines on synthetic data as well as real-world benchmarks, for tasks involving prediction and influencing set identification.

Although pairwise causal relations have been extensively studied in observational longitudinal analyses across many disciplines, incorporating knowledge of causal pairs into deep learning models for temporal event sequences remains largely unexplored. In this paper, we propose a novel approach for enhancing the performance of transformer-based models in multivariate event sequences by injecting pairwise qualitative causal knowledge such as `event Z amplifies future occurrences of event Y'. We establish a new framework for causal inference in temporal event sequences using a transformer architecture, providing a theoretical justification for our approach, and show how to obtain unbiased estimates of the proposed measure. Experimental results demonstrate that our approach outperforms several state-of-the-art models in terms of prediction accuracy by effectively leveraging knowledge about causal pairs. We also consider a unique application where we extract knowledge around sequences of societal events by generating them from a large language model, and demonstrate how a causal knowledge graph can help with event prediction in such sequences. Overall, our framework offers a practical means of improving the performance of transformer-based models in multivariate event sequences by explicitly exploiting pairwise causal information.

Complex care management (CCM) or hot spotting programs identify and manage high-need/high-cost patients, improving long-term health quality and medical costs. Typically, physicians refer patients to CCM. Despite strict guidelines to ensure that eligible patients are placed in appropriate programs, such a provider-based approach is limited by provider-capacity and the narrow view of a patient that a provider sees. We propose an ML workflow to augment the provider-based approach, that can flag patients who are suited to CCM. Our predictor uses a global view of a patient’s entire history across multiple providers and time to identify high-risk individuals from among all the individuals in a matter of seconds. On a monthly basis, we evaluate our predictions against physician referrals. In the test dataset, 41% of the top-500 highest risk individuals found by our model were referred to CCM by a physician at some point in the 6-month window following our prediction (top-500 is a parameter that can be set to match the CCM program’s capacity). Of those who were not referred in the 6-month window, 30% were referred at some time in their trajectory. The remaining false positives had a greater than 95% similarity when compared to true positive physician referrals in terms of cost profiles (both prior to referral and after referral) and patient profile. This remarkable similarity suggests that our machine learning predictor can identify new candidates for complex care management and/or predict referrals before a physician has an opportunity to do so.

Causal inference and discovery from observational data has been extensively studied across multiple fields. However, most prior work has focused on independent and identically distributed (i.i.d.) data. In this paper, we propose a formalization for causal inference between pairs of event variables in multivariate recurrent event streams by extending Rubin's framework for the average treatment effect (ATE) and propensity scores to multivariate point processes. Analogous to a joint probability distribution representing i.i.d. data, a multivariate point process represents data involving asynchronous and irregularly spaced occurrences of various types of events over a common timeline. We theoretically justify our point process causal framework and show how to obtain unbiased estimates of the proposed measure. We conduct an experimental investigation using synthetic and real-world event datasets, where our proposed causal inference framework is shown to exhibit superior performance against a set of baseline pairwise causal association scores.

We examine a machine learning approach for deriving insights from observational healthcare data in order to improve public health. Our goal is to simultaneously identify patient subpopulations with differing health risks and find the distinct risk factors or determinants associated with each subpopulation. Here, we develop a supervised Gaussian Mixture Model (GMM) approach for subpopulation modeling that combines GMMs with L1-logistic regression. We demonstrate the approach on an analysis of high cost drivers of Medicaid expenditures for inpatient stays associated with Newborn, Pregnancy, and Circulatory Systems diagnostic categories. These conditions were chosen because they had the highest total inpatient expenditures in New York State (NYS) in 2016. When compared with state-of-the-art learning methods (random forests, boosting, deep learning), our approach provides comparable prediction performance but also extracts insightful explanations of the subpopulation structure and risk factors within each subpopulation. Sequentially applying unsupervised learning methods and then applying logistic regression fails to yield equally meaningful results: the unsupervised subpopulations are homogeneous and moderately predictable, while some of our subpopulations are highly predictable with easy-to-identify drivers of cost. Focusing on newborns, we unveil subpopulations indicative of the landscape of healthcare in NYS: about 90% of the discharges are healthy New York City babies and about 1% are costly complex cases. Subpopulations indicate regional disparities: for example newborns from Central, Southern and Western NY are of higher risk for high-cost stays associated with substance abuse. The results indicate the promise of the approach for future population health studies based on electronic health care records.