Primary Submission Category: Design-Based Causal Inference
Design-based Inference with the Estimated Propensity Score
Authors: Shunzhuang Huang, Jiangchuan Du, Azeem Shaikh, Panos Toulis,
Presenting Author: Shunzhuang Huang*
In this paper, we study the properties of design-based (or randomization) inference for treatment effects when analyzing observational data under ignorability. In such settings, we interpret the common ignorability assumption as defining an artificial randomized experiment and study approximate randomization tests that use the estimated propensity score, i.e., the distribution of treatment status given the covariates, as a foundation for design-based inference. Under the sharp null hypothesis of no treatment effect in distribution, we derive non-asymptotic bounds on the size distortion of such tests that depend only on the error in estimating the propensity score. Under the weak null hypothesis of no average treatment effect, we show that the proposed tests are asymptotically valid for common estimators, including inverse-propensity-weighted and doubly-robust estimators. We further compare our tests with conventional tests based on the asymptotic normality for the weak null hypothesis and, since these tests are shown to be first-order equivalent, develop higher-order comparisons using novel Edgeworth expansions. Our analysis reveals that, from this perspective, neither approach uniformly dominates the other. However, the randomization test for the weak null achieves higher-order accuracy when the sharp null “nearly holds”; for example, when treatment effects are small or rare.