Primary Submission Category: Difference in Differences
Practical Guidance on Whether and When to Aggregate Individual-Level Data for Causal Health Policy Evaluation
Authors: Nicholas Seewald, Kayla Tormohlen, Beth McGinty, Elizabeth Stuart,
Presenting Author: Nicholas Seewald*
Health policy researchers often have questions about the effects of state policy on individual-level outcomes collected over multiple time periods. Such questions might be addressed using, for example, a large health insurance claims database that tracks individuals’ receipt of a particular treatment. An open question is whether the researcher can or should “roll-up” (i.e., aggregate, average, etc.) this individual-level data to the policy level when assessing the effects of state policy. Rolling up the data offers a clear computational advantage since it makes the individual-level big data question much smaller. However, existing literature does not sufficiently address whether and when aggregation is disadvantageous due to loss of individual-level information. Here, we examine the statistical performance of difference-in-differences approaches that permit the use of either individual- or aggregate-level data to offer practical guidance on whether and when to roll up. Our guidance is based on simulation models which allow us to make fair comparisons between analytic methods under a variety of controlled conditions. We also discuss our recommendations in the context of a study designed to assess the effects of state medical cannabis laws on opioid prescribing among patients with chronic non-cancer pain.