Primary Submission Category: Causal Inference and SUTVA/Consistencies Violations
Switchback Experiments under Geometric Mixing
Authors: Yuchen Hu, Stefan Wager,
Presenting Author: Yuchen Hu*
The switchback is an experimental design that measures treatment effects by repeatedly turning an intervention on and off for a whole system. Switchback experiments are a robust way to overcome cross-unit spillover effects; however, they are vulnerable to bias from temporal carryovers. In this paper, we consider properties of switchback experiments in Markovian systems that mix at a geometric rate. We find that, in this setting, standard switchback designs suffer considerably from carryover bias: Their estimation error decays as T^{-1/3} in terms of the experiment horizon T, whereas in the absence of carryovers a faster rate of T^{-1/2} would have been possible. We also show, however, that judicious use of burn-in periods can considerably improve the situation, and enables errors that decay almost as fast as T^{-1/2}. Our formal results are mirrored in an empirical evaluation.