Primary Submission Category: Sensitivity Analysis
Sensitivity analysis for null results: Implications for studies of racially biased policing
Authors: Jake Bowers, Tom Leavitt, Luke Miratrix,
Presenting Author: Jake Bowers*
We propose a method of formal sensitivity analysis for causal inference that addresses the problem of understating rather than overstating causal effects. A null result in an observational study is no more or less likely to emerge because of hidden confounding than a strong result. We motivate this work with the problem of statistically and substantively insignificant results in the study of the causal effects of race of civilian on police use of force and show how it adds to existing critiques of null results. We build on existing criticisms of naive estimation of the effect of race on police uses of force, adding a sensitivity analysis that addresses the possibilty that a given result understates the true effect both becuase of a pattern of hidden confounding and also a pattern of post-treatment missingness like that seen in datasets use to study race and police can combine to produce a misleading null effect. And we show how our method of sensitivity analysis for null effects reveals that the null result is, in fact, sensitive to these kinds of bias.