Primary Submission Category: Design-Based Causal Inference
Data-driven hypotheses and designs for theory testing and theory building
Authors: Molly Offer-Westort,
Presenting Author: Molly Offer-Westort*
This project develops a design-based framework for theory testing when a theory admits multiple empirical realizations, and theory building when multiple alternative mechanisms are under consideration. Experiments in the social sciences often test a single treatment arm against a control, or test the effects of multiple treatments independently. However, many social-scientific mechanisms can be implemented through a family of related interventions. This is formalized by indexing a treatment space W and defining data-driven hypotheses such as an EXISTENCE (maxw∈W θ (w) ≤c) or UNIVERSAL (minw∈W θ (w) ≤c) hypothesis. For these types of hypotheses, data-driven explore-confirm designs that separate learning from inference can improve power. A simple power factorization is used to compare uniform, cross-fit, and data adaptive designs. Simple two-stage data-adaptive designs improve power by concentrating confirmatory allocation to the treatment(s) most likely to yield rejection under the alternative. Adaptive first stage exploration improves selection quality. The framework links selective inference, adaptive experimentation, and mechanism testing, providing a general approach to data-driven design.