Abstract Causal discovery from observational data alone in the presence of unobserved common causes is crucial yet challenging. We categorize the causal relationship between two random variables $$X$$ and $$Y$$ into the following four categories: two direct-case cases ( $$X \rightarrow Y$$ or $$X \leftarrow Y$$ ), a case in which $$X$$ and $$Y$$ are independent, and a case in which $$X$$ and $$Y$$ are confounded by unobserved common causes $$C$$ , and aim to decide among them from observational data alone. We call this the Reichenbach problem, since this categorization is valid if the Reichenbach’s common cause principle is assumed to be true. Although many methods have been proposed to address causal structure learning in the presence of $$C$$ , they typically impose assumptions on the nature of $$C$$ that require knowledge of how unobserved confounders behave, and it is difficult to guarantee in many practical cases compared to assumptions about observed variables. In our previous study (Kobayashi et al. in: 2022 IEEE international conference on big data (Big Data). IEEE, pp 45–54, 2022), we proposed a causal discovery method without such assumptions regarding the nature of unmeasured confounders, named $${\mathbf{\mathsf{CLOUD}}}$$ , for discrete data. Building on the Reichenbach’s common cause principle, if neither the two direct-cause models nor the independence model can explain the data well, we can infer the involvement of unobserved common causes in generating $$X$$ and $$Y$$ . To realize this idea in practice, $$\mathbf{\mathsf{CLOUD}}$$ imposes structural assumptions on the direct-cause models such as additive noise models ( $$\mathbf{\mathsf{ANM}}$$ s) (Peters et al. in: Proceedings of the thirteenth international conference on artificial intelligence and statistics. JMLR workshop and conference proceedings, pp 597–604, 2010), while allowing the confounded model to represent any joint distribution on ( $$X$$ , $$Y$$ ) by placing no assumptions on $$C$$ . $$\mathbf{\mathsf{CLOUD}}$$ then employs the normalized maximum likelihood (NML) codelength (Shtar’kov in Problemy Peredachi Informatsii 23(3):3–17, 1987) as an information criterion and compares models of varying capacities. This study extends $$\mathbf{\mathsf{CLOUD}}$$ to include all data types. We provide NML formulations and theoretical guarantees for consistency in model selection for mixed and continuous cases. Through extensive experiments on both synthetic and real-world data, we demonstrated that $$\mathbf{\mathsf{CLOUD}}$$ is more effective than existing methods in inferring causal relationships.

A longstanding goal in safe reinforcement learning (RL) is a method to ensure the safety of a policy throughout the entire process, from learning to operation. However, existing safe RL paradigms inherently struggle to achieve this objective. We propose a method, called Provably Lifetime Safe RL (PLS), that integrates offline safe RL with safe policy deployment to address this challenge. Our proposed method learns a policy offline using return-conditioned supervised learning and then deploys the resulting policy while cautiously optimizing a limited set of parameters, known as target returns, using Gaussian processes (GPs). Theoretically, we justify the use of GPs by analyzing the mathematical relationship between target and actual returns. We then prove that PLS finds near-optimal target returns while guaranteeing safety with high probability. Empirically, we demonstrate that PLS outperforms baselines both in safety and reward performance, thereby achieving the longstanding goal to obtain high rewards while ensuring the safety of a policy throughout the lifetime from learning to operation.

We propose a method of offline reinforcement learning (RL) featuring the performance guarantee without any assumptions on the data support. Under such conditions, estimating or optimizing the conventional performance metric is generally infeasible due to the distributional discrepancy between data and target policy distributions. To address this issue, we employ a worst-case policy value as a new metric and constructively show that the sample complexity bound of $O(\epsilon^{−2})$ is attainable without any data-support conditions, where $\epsilon>0$ is the policy suboptimality in the new metric. Moreover, as the new metric generalizes the conventional one, the algorithm can address standard offline RL tasks without modification. In this context, our sample complexity bound can be seen as a strict improvement on the previous bounds under the single-policy concentrability and the single-policy realizability.

We are interested in an evaluation methodology for molecular optimization. Given a sample of molecules and their properties of our interest, we wish not only to train a generator of molecules optimized with respect to a target property but also to evaluate its performance accurately. A common practice is to train a predictor of the target property using the sample and apply it to both training and evaluating the generator. However, little is known about its statistical properties, and thus, we are not certain about whether this performance estimate is reliable or not. We theoretically investigate this evaluation methodology and show that it potentially suffers from two biases; one is due to misspecification of the predictor and the other to reusing the same finite sample for training and evaluation. We discuss bias reduction methods for each of the biases, and empirically investigate their effectiveness.

Inference of causal structure in random variables from observed data only is a crucial science problem. This study categorizes the causal relationship between two discrete random variables into four categories, which include the case that variables have a common cause and that variables are independent in addition to two directions of the direct causality. Although several existing methods have been proposed for causal inference from a joint distribution of two discrete random variables that can select either direction of the direct causality, methods that can directly infer a causal relationship from the four categories mentioned are limited. We proposed the first method to infer a causal relationship without any assumption on the unobserved confounder in discrete data. Based on the minimum description length (MDL) principle, the proposed method calculates the code length of the observed data for each causal model and then selects a model which yields the minimum code length. We demonstrated that the proposed method is effective in detecting common causes and outperforms the existing methods in terms of the accuracy of inference using synthetic data. We further showed that the proposed method effectively detects an unobserved confounder in real-world data.

We address the problem of predicting when a disease will develop, i.e., medical event time (MET), from a patient's electronic health record (EHR). The MET of non-communicable diseases like diabetes is highly correlated to cumulative health conditions, more specifically, how much time the patient spent with specific health conditions in the past. The common time-series representation is indirect in extracting such information from EHR because it focuses on detailed dependencies between values in successive observations, not cumulative information. We propose a novel data representation for EHR called cumulative stay-time representation (CTR), which directly models such cumulative health conditions. We derive a trainable construction of CTR based on neural networks that has the flexibility to fit the target data and scalability to handle high-dimensional EHR. Numerical experiments using synthetic and real-world datasets demonstrate that CTR alone achieves a high prediction performance, and it enhances the performance of existing models when combined with them.

Partially monotone regression is a regression analysis in which the target values are monotonically increasing with respect to a subset of input features. The TensorFlow Lattice library is one of the standard machine learning libraries for partially monotone regression. It consists of several neural network layers, and its core component is the lattice layer. One of the problems of the lattice layer is that it requires the projected gradient descent algorithm with many constraints to train it. Another problem is that it cannot receive a high-dimensional input vector due to the memory consumption. We propose a novel neural network layer, the hierarchical lattice layer (HLL), as an extension of the lattice layer so that we can use a standard stochastic gradient descent algorithm to train HLL while satisfying monotonicity constraints and so that it can receive a high-dimensional input vector. Our experiments demonstrate that HLL did not sacrifice its prediction performance on real datasets compared with the lattice layer.

We consider the problem of offline policy evaluation~(OPE) with Markov decision processes~(MDPs), where the goal is to estimate the utility of given decision-making policies based on static datasets. Recently, theoretical understanding of OPE has been rapidly advanced under (approximate) realizability assumptions, i.e., where the environments of interest are well approximated with the given hypothetical models. On the other hand, the OPE under unrealizability has not been well understood as much as in the realizable setting despite its importance in real-world applications.To address this issue, we study the behavior of a simple existing OPE method called the linear direct method~(DM) under the unrealizability. Consequently, we obtain an asymptotically exact characterization of the OPE error in a doubly robust form. Leveraging this result, we also establish the nonparametric consistency of the tile-coding estimators under quite mild assumptions.

We approach the time-series forecasting problem in the presence of concept drift by automatic learning rate tuning of stochastic gradient descent (SGD). The SGD-based approach is preferable to other concept drift algorithms in that it can be applied to any model and it can keep learning efficiently whilst predicting online. Among a number of SGD algorithms, the variance-based SGD (vSGD) can successfully handle concept drift by automatic learning rate tuning, which is reduced to an adaptive mean estimation problem. However, its performance is still limited because of its heuristic mean estimator. In this paper, we present a concept-drift-aware stochastic gradient descent (Cogra), equipped with more theoretically-sound mean estimator called sequential mean tracker (SMT). Our key contribution is that we define a goodness criterion for the mean estimators; SMT is designed to be optimal according to this criterion. As a result of comprehensive experiments, we find that (i) our SMT can estimate the mean better than vSGD’s estimator in the presence of concept drift, and (ii) in terms of predictive performance, Cogra reduces the predictive loss by 16–67% for real-world datasets, indicating that SMT improves the prediction accuracy significantly.

We develop a new theoretical framework, the envelope complexity, to analyze the minimax regret with logarithmic loss functions. Within the framework, we derive a Bayesian predictor that adaptively achieves the minimax regret over high-dimensional l1-balls within a factor of two. The prior is newly derived for achieving the minimax regret and called the spike-and-tails (ST) prior as it looks like. The resulting regret bound is so simple that it is completely determined with the smoothness of the loss function and the radius of the balls except with logarithmic factors, and it has a generalized form of existing regret/risk bounds.

This study addresses the problem of detecting change points in streaming data. Herein, we focus on detecting changes with low complexity and high accuracy to manage large-volume, high-velocity data. To this end, we propose a novel method of detecting changes via data compression, by introducing minimum description length (MDL) change statistics to an adaptive windowing regime. The time complexity of the resulting algorithm, Sequential Compression with Adaptive Windowing (SCAW), is optimal except for a logarithmic factor, and is theoretically justified through exponential upper bounds on error probabilities. Moreover, SCAW can be used to detect arbitrary types of changes by choosing an appropriate compressor. We also introduce the notion of asymptotic reliability as a criterion of change point detection algorithms, and determine SCAW's parameter using this criterion. Finally, we demonstrate the effectiveness of the proposed method in experiments using synthetic and real-world data from markets and industrial machinery.

Discovering a true sparse model capable of generating data is a challenging yet important problem for understanding the nature of the source of the data. A major part of the challenge arises from the fact that the number of possible sparse models grows exponentially as the dimensionality of the models increases. In this study, we consider a method for estimating the true model over an exponentially large number of sparse models based on the minimum description length principle. We show that a novel criterion derived by continuous relaxation of the stochastic complexity induces selection of the true model by solving the l1-regularization problem for which the hyperparameters are appropriately chosen. Moreover, we provide an efficient optimization algorithm for finding the appropriate hyperparameters and select the sparse model accordingly. The experimental results we obtained for the problem of sparse graphical modeling indicate that the proposed method estimates the true model effectively in comparison to existing methods for choosing hyperparameters to solve the l1-regularization problem.

In this paper we propose a novel methodology of sequential change detection using the minimum description length (MDL)-change statistics. We first introduce the MDL-change statistics as the difference between the code-lengths with change and that without change. We give a theoretical justification for its use in the scenario of hypothesis testing. In it we evaluate the error probabilities for the MDL-change detection to relate them to the information-theoretic complexities of the probabilistic models and their discrepancy measure. We then convert the MDL-change statistics into the sequential change detection algorithm. It is designed to detect gradual changes as well as abrupt changes from big stream data. We empirically demonstrate the effectiveness of the proposed method by showing that it performs better than existing algorithms for synthetic data. We also show its validity through real problems such as SQL injection detection and failure symptom detection.

Convolutive non-negative matrix factorization (CNMF) is a promising method for extracting features from sequential multivariate data. Conventional algorithms for CNMF require that the structure, or the number of bases for expressing the data, be specified in advance. We are concerned with the issue of how we can select the best structure of CNMF from given data. We first introduce a framework of probabilistic modeling of CNMF and reduce this issue to statistical model selection. The problem is here that conventional model selection criteria such as AIC, BIC, MDL cannot straightforwardly be applied since the probabilistic model for CNMF is irregular in the sense that parameters are not uniquely identifiable. We overcome this problem to propose a novel criterion for best structure selection for CNMF. The key idea is to apply the technique of latent variable completion in combination with normalized maximum likelihood coding criterion under the minimum description length principle. We empirically demonstrate the effectiveness of our method using artificial and real data sets.

This paper addresses the issue of detecting changes in stochastic processes. In conventional studies on change detection, it has been explored how to detect discrete changes for which the statistical models of data suddenly change. We are rather concerned with how to detect continuous changes which occurs incrementally over some successive periods. This paper gives a novel methodology for detecting continuous changes. We first define the information-theoretic measure of continuous change and prove that it is invariant with respect to the parametrization of statistical model. We then propose an efficient algorithm to detect continuous changes according to the proposed measure. We demonstrate the effectiveness of our method through the experiments using synthetic data and the applications to security and economic event detection.