Primary Submission Category: Simulation Methods
Simulation Design as Causal Intervention on a Data Generating Mechanism
Authors: Tyrel Stokes, Russell Steele, Ian Shrier,
Presenting Author: Tyrel Stokes*
Simulation methods are among the most ubiquitous methodological tools in statistical science. In particular, simulation is commonly used to explore properties of statistical functionals in models for which developed statistical theory is insufficient or to assess finite sample properties of theoretical results. We show that simulation experimental design can be viewed from the perspective of causal intervention on a data generating mechanism and demonstrate the use of causal tools and frameworks in this context. Most notably, our perspective is agnostic to the particular domain of the simulation experiment which increases the potential impact of our proposed approach. In this talk, we consider two illustrative examples. First, we re-examine a predictive machine learning example from a popular textbook designed to assess the relationship between mean function complexity and the mean-squared error. Second, we discuss a traditional causal inference method problem, simulating the effect of unmeasured confounding on estimation, specifically to illustrate bias amplification. In both cases, applying causal principles and using graphical models with parameters and distributions as nodes in the spirit of influence diagrams can 1) make precise the estimand the simulation targets , 2) suggest modifications to better attain the simulation goals, and 3) provide scaffolding to discuss performance criteria for a particular simulation design, such as the generalizability to other contexts.