Primary Submission Category: Difference in Differences, Synthetic Control, Methods for Panel and Longitudinal Data
Small-Sample Performance of Synthetic Difference-in-Differences
Authors: Luke Stewart, Nandita Mitra, Youjin Lee, Gary Hettinger,
Presenting Author: Luke Stewart*
Methods for estimating causal effects in longitudinal, quasi-experimental settings are widely used in economics, public health, and other fields. The recently proposed synthetic difference-in-differences (SDiD) method improves on difference-in-differences (DiD) and synthetic controls (SC) by weakening the conditions necessary to identify and estimate a causal effect in settings with longitudinal panel data or repeated cross sectional data. However, these studies often have limited sample sizes, both in terms of units and periods analyzed. Although SDiD generally relies on weaker assumptions than both DiD and SC, the required asymptotics may restrict the use of the method.
To evaluate the performance of SDiD in realistic, small-sample settings, we employ two complementary simulation strategies. First, we reanalyze existing placebo control simulation studies while artificially limiting the number of units and periods. Second, we introduce a novel simulation framework for panel data that allows the injection of a known treatment effect into existing data in a setting of interest. We simulate data under small samples and assess the robustness of SDiD to violations of its assumptions about the error term in the latent factor model. Drawing on findings from these investigations, we offer guidance for researchers on when SDiD is likely to yield reliable estimates. Finally, we apply SDiD to an investigation of the effect of the Philadelphia Beverage Tax on youth soda consumption.