Primary Submission Category: Difference in Differences
Handling Correlation in Stacked Difference-in-Differences Estimates with Application to Medical Cannabis Policy
Authors: Nicholas Seewald, Beth McGinty, Kayla Tormohlen, Elizabeth Stuart,
Presenting Author: Nicholas Seewald*
Health policy researchers often have questions about the effects of state policy on individual-level outcomes collected over multiple time periods. In some cases, these studies are conducted using large-scale, individual-level administrative data such as health insurance claims. However, there are multiple open questions about the use of such individual-level data in difference-in-differences (DiD) analyses. “Stacked” DiD is one approach to estimate treatment effects when units implement the policy of interest at different times: for each unit enacting a policy, we construct a comparison group of units that never enact (or had not yet enacted) the policy, analyze each treated unit separately with its comparison group, then pool effect estimates. However, when individual-level data is available, some individuals in untreated units can contribute to comparison groups for multiple treated states, producing correlation between stacked estimates. Existing methods do not quantify or account for this sharing of controls: this leads to incorrect inference. Here, we present a framework for estimating and managing this correlation when pooling stacked effect estimates. We explore the statistical properties of this approach, examine its performance in realistic simulations and with real data, and explain its occasionally counterintuitive effects on variance estimates. This is motivated by a study investigating the effects of state medical cannabis laws on opioid prescribing for pain.