Primary Submission Category: Design of Experiments
Maximizing Gains from Existing Information: An Adaptive Pairing and Stratification Procedure for Experimental Designs
Authors: Zikai Li,
Presenting Author: Zikai Li*
In social science experiments, given a fixed budget and/or a limited number of participants, researchers need to optimize statistical efficiency. To this end, they often use stratification. However, conventional practices of stratification are agnostic with respect to the predictive relationship between the covariates and the outcomes, even though it is this predictive relationship that motivates stratification. Such practices thus fail to take advantage of all available information when some data is available for the experimental outcome and the covariates. This paper introduces an adaptive pairing and stratification procedure for running an experiment in batches. This approach builds upon recent work that demonstrates the theoretical optimality of pairing units based on the expected sum of potential outcomes. The method incorporates information about the relationship between covariates and potential outcomes when pairing or stratifying units. It uses data from earlier batches not just to inform pairing decisions but also to rematch observations across different batches without compromising the validity of inference under the superpopulation framework. In experimental settings where sequential treatment assignment and outcome collection are feasible, this approach can improve the efficiency of treatment assignments relative to its alternatives. My simulations demonstrate such gains can be substantial.