Primary Submission Category: Causal Inference and SUTVA/Consistencies Violations
Design of Panel Experiments with Spatial and Temporal Interference
Authors: Tu Ni, Iavor Bojinov, Jinglong Zhao,
Presenting Author: Tu Ni*
Panel experiments — where we expose multiple units to some random treatments, measure their responses, and repeat the procedure for some time periods — have rapidly grown popular in marketplace companies, wishing to run randomized controlled experiments (A/B testing) in the presence of spatial interference between experimental units and temporal interference between time periods.
When running the experiments, companies group units together across zip codes, cities, or even states to form a single aggregated unit, in order to alleviate the spatial interference between units, as it ensures that each unit within the aggregated unit receives the same treatment, but it does not remove the temporal interference over time.
Unfortunately, such a drastic aggregation significantly reduces the sample size, leading to much lower power for inference. This highlights a critical trade-off when a panel experiment has interference: aggregation limits the degree of interference but reduces the volume of data.
In this work, we examine this trade-off and present a new, more powerful, randomized design for panel experiments in the presence of spatial and temporal interference.
Our proposed design has two features: the first feature is a notion of cluster-based randomization that allows us to navigate the aforementioned trade-off for aggregation transparently via the cluster size; the second feature is a notion of balanced randomization of treatment and control that incorporates an assign