Anytime-valid simulation-based hypothesis testing

arXiv:2610.08210v1 Announce Type: cross Abstract: For a given data distribution (X_t){t in N} sim Q i.i.d., we investigate the hypothesis testing problem: H_0: Q = P_0 vs. H_1: Q = P_1, for two different model probability distributions P_0 and P_1. In contrast to the standard setting, where analytic densities p_0 and p_1 are given, here, we consider the density-free setting, where we only have access to i.i.d. simulations (Z^0_t){t in N} sim P_0 and (Z^1_t)_{t in N} sim P_1. For this simulation-based hypothesis testing setting, we construct an e-test martingale, resulting in a sequential test with anytime-valid type-I error guarantees, approximate growth optimality, geometrically decaying type-II error bounds, and asymptotic power one. Most ingredients used in our constructions are variants of well known concepts. The value of this paper lies in the compact presentation of an effective, anytime-valid solution for the density-free simulation-based sequential hypothesis testing case.

Related

• Optimal Rates for Differentially Private Hypothesis Testing with E-values
• Type-II Error Bounds for Test Supermartingales from Lower-Tail Hypotheses
• The Type-II Error of Test Supermartingales: e-Power versus the Chernoff-Stein Exponent
• Semi-Supervised Hypothesis Testing by Betting on Predictions
• The Price of Peeking: Anytime-Valid Leakage Detection on ML-KEM EM Traces

Source: arXiv cs.LG | 2026-10-07