Primary Submission Category: Generalizability/Transportability
Selective Experimentation and Stability Radius Models
Authors: Adam Bouyamourn,
Presenting Author: Adam Bouyamourn*
Choosing an internally-valid causal inference design is not sufficient to ensure that the conclusions of a paper are true: researchers may strategically select experiments whose conclusions do not match the inferences that would have been drawn if a larger or more representative experiment was conducted. Using a formal model to study researcher incentives, I first show that the problem is one of information revelation: the problem is that honest researchers cannot credibly communicate that they did not strategically select their experiment. Drawing inspiration from the control theory literature on stability radius models, I then show that reporting a stability radius can induce sufficient information disclosure to allow an audience to assess whether or not to trust the conclusion of a given paper. Then, I develop two empirical tools for estimating stability radii, each using a conformal wrapper for inference: the first using Support Vector Machines, the second using a factor model.