Statistical testing under distributional shifts

Research output: Contribution to journalJournal articleResearchpeer-review


  • Fulltext

    Final published version, 4.69 MB, PDF document

We introduce statistical testing under distributional shifts. We are interested in the hypothesis
for a target distribution
⁠, but observe data from a different distribution
⁠. We assume that
is related to
through a known shift τ and formally introduce hypothesis testing in this setting. We propose a general testing procedure that first resamples from the observed data to construct an auxiliary data set (similarly to sampling importance resampling) and then applies an existing test in the target domain. We prove that if the size of the resample is of order
and the resampling weights are well behaved, this procedure inherits the pointwise asymptotic level and power from the target test. If the map τ is estimated from data, we maintain the above guarantees under mild conditions on the estimation. Our results extend to finite sample level, uniform asymptotic level, a different resampling scheme, and statistical inference different from testing. Testing under distributional shifts allows us to tackle a diverse set of problems. We argue that it may prove useful in contextual bandit problems and covariate shift, show how it reduces conditional to unconditional indep
Original languageEnglish
JournalJournal of the Royal Statistical Society, Series B (Statistical Methodology)
Issue number3
Pages (from-to)597-663
Publication statusPublished - 2023

Number of downloads are based on statistics from Google Scholar and

No data available

ID: 370486535