In two-sampling testing, one observes two independent sequences of independent and identically distributed random variables distributed according to the distributions $P_1$ and $P_2$ and wishes to decide whether $P_1=P_2$ (null hypothesis) or $P_1\neq P_2$ (alternative hypothesis). The Gutman test for this problem compares the empirical distributions of the observed sequences and decides on the null hypothesis if the Jensen-Shannon (JS) divergence between these empirical distributions is below a given threshold. This paper proposes a generalization of the Gutman test, termed \emph{divergence test}, which replaces the JS divergence by an arbitrary divergence. For this test, the exponential decay of the type-II error probability for a fixed type-I error probability is studied. First, it is shown that the divergence test achieves the optimal first-order exponent, irrespective of the choice of divergence. Second, it is demonstrated that divergence tests with invariant divergences achieve the same second-order asymptotics as the Gutman test. In addition, a connection between two-sample testing and robust goodness-of-fit testing is established.

Consider a composite hypothesis testing problem where the test has access to the null hypothesisPbut not to the alternative hypothesisQ. The generalized likelihood-ratio test (GLRT) for this problem is the Hoeffding test, which acceptsPif the Kullback-Leibler (KL) divergence between the empirical distribution ofZnandPis below some threshold. This paper proposes a generalization of the Hoeffding test, termed divergence test, for which the KL divergence is replaced by an arbitrary divergence. For this test, the first and second-order terms of the type-II error probability for a fixed type-I error probability are characterized and compared with the error terms of the Neyman-Pearson test, which is the optimal test when bothPandQare known. It is demonstrated that, irrespective of the divergence, divergence tests achieve the first-order term of the Neyman-Pearson test. In contrast, the second-order term of divergence tests is strictly worse than that of the Neyman-Pearson test. It is further demonstrated that divergence tests with an invariant divergence achieve the same second-order term as the Hoeffding test, but divergence tests with a non-invariant divergence may outperform the Hoeffding test for some alternative hypothesesQ. This implies that the GLRT may have a second-order asymptotic performance that is strictly suboptimal.

We consider a binary statistical hypothesis testing problem, where n independent and identically distributed random variables Znare either distributed according to the null hypothesis P or the alternate hypothesis Q, and only P is known. For this problem, a well-known test is the Hoeffding test, which accepts P if the Kullback-Leibler (KL) divergence between the empirical distribution of Znand P is below some threshold. In this paper, we consider Hoeffding-like tests, where the KL divergence is replaced by other divergences, and characterize, for a large class of divergences, the first and second-order terms of the type-II error for a fixed type-I error. Since the considered class includes the KL divergence, we obtain the second-order term of the Hoeffding test as a special case.