Statistical Policy Comparison (SPC) assesses the equivalence of two stochastic policies (policy consistency) and has received broad attention. However, the SPC framework implicitly assumes the invariance of decision environments, and therefore fails to address a flurry of real-world data science applications. In this work, we refer to this overlooked issue as environment consistency, and together with policy consistency, this extends to a generalized concept process consistency for systematically comparing policy trials under the Markov decision process (MDP) framework. To address process consistency, we propose a unified comparison framework, extending beyond traditional statistical policy comparison studies by incorporating both policy and environment comparisons. For policy consistency, existing statistical policy comparison methods can be seamlessly integrated into our intentionally-designed framework without modification. Specifically for environment consistency (the focus of this work), we devise fine-grained return tests to capture shifts of key elements in MDPs; notably, under special cases where trajectory likelihood information is available or can be estimated, we introduce a trajectory test based on the likelihood ratio test (LRT), offering increased testing power. Extensive experiments demonstrate that our proposed testing methods achieve higher statistical power than existing approaches in testing process consistency, establishing their effectiveness across diverse real-world scenarios. Our code is available at https://github.com/bcxyf123/MDP-Testing.git.

The problem of testing a finite number of possibly composite hypotheses is considered, when it is required to control the probability of every type of wrong decision below a user-specified level. A general strategy is proposed for constructing a 3-stage test, in which the size of the second stage depends on the data collected in the first stage. Sufficient conditions are established for such an adaptive 3-stage test to achieve the optimal expected sample size, among all admissible sequential tests, to a first-order asymptotic approximation as the error probabilities go to zero at relatively symmetric rates. This general framework is applied to the case of general simple hypotheses, where the data are not necessarily i.i.d., as well as to the case of composite hypotheses of i.i.d. data coming from a one-parameter exponential family.

A multistage test is proposed for the problem of testing an arbitrary number of simple hypotheses regarding the distribution of a sequence of i.i$\mathbf{d}$. random elements. The proposed test is shown to control the probability of each possible error under an arbitrary, user-specified level. Most importantly, it is shown to achieve the optimal expected sample size under every hypothesis, in the class of all sequential tests with the same levels of error control, to a first-order asymptotic approximation as these levels go to zero. These theoretical results are illustrated in a simulation study, where the proposed multistage test is compared with an asymptotically optimal fully-sequential test.

The problem of simultaneously testing the distributions of a large number of independent, sequentially observed data streams is considered, where the same multiple hypotheses are posed for each data stream, and the goal is to correctly identify all data streams following each hypothesis. A decentralized setup is adopted, under which the testing procedure applied to each data stream must be the same and can only use local observations. A novel criterion of high-dimensional asymptotic optimality is proposed, according to which the goal is to achieve the optimal expected average sample size, uniformly in all possible hypothesis configurations, asymptotically as the total number of data streams and the maximum possible numbers of data streams following each hypothesis go to infinity, in the class of all testing procedures that control the same levels of familywise error rates. We show that this criterion is achieved by the multihypothesis sequential probability ratio test.

A signal recovery problem is considered, where the same binary testing problem is posed over multiple, independent data streams. The goal is to identify all signals (resp. noises), i.e., streams where the alternative (resp. null) hypothesis is correct, subject to prescribed bounds on classical or generalized familywise error probabilities of both types. It is not required that the exact number of signals be a priori known, only upper bounds on the numbers of signals and noises are assumed instead. A decentralized formulation is adopted, according to which the sample size and the decision for each testing problem must be based only on observations from the corresponding data stream. A novel multistage testing procedure is proposed for this problem and is shown to enjoy a high-dimensional asymptotic optimality property. Specifically, it achieves the optimal, average over all streams, expected sample size, uniformly in the true number of signals, as the maximum possible numbers of signals and noises go to infinity at arbitrary rates, in the class of all sequential tests with the same global error control. In contrast, existing multistage tests in the literature are shown to achieve this high-dimensional asymptotic optimality property only under additional sparsity or symmetry conditions. These results are based on an asymptotic analysis for the fundamental binary testing problem as the two error probabilities go to zero. Moreover, they are supported by simulation studies and extended to problems with non-iid data and composite hypotheses.

The problem of testing two simple hypotheses about the distribution of iid random elements is considered. In particular, the focus is on multistage tests that control the two error probabilities below arbitrary, user-specified levels. A novel multistage test is proposed, analyzed, and shown to achieve the optimal expected sample size under both hypotheses, in the class of all sequential tests with the same error control, to a first-order approximation as the two target error probabilities go to zero at arbitrary rates. The proposed test is compared, both theoretically and numerically, with a multistage test that enjoys the same asymptotic optimality property under one of the two hypotheses, while performing much worse under the other.