Primary Submission Category: Causal Inference and Bias/Discrimination
A Decision-Theoretic Framework for Sample Selection in Randomized Experiments
Authors: Yuchen Hu, Stefan Wager, Emma Brunskill, Henry Zhu,
Presenting Author: Yuchen Hu*
The design of randomized experiments often fails to account for heterogeneity of treatment effects across different subpopulations, and discussions on how to reflect different fairness-oriented desiderata in study design are largely absent from the literature. For example, until recently, most medical research in the United States was conducted on white men, while excluding women and racial minorities (e.g., Dresser, 1992, “Wanted single, white male for medical research”); and FDA-approved trials still under-sample black participants relative to their share of the population (Alsan, EAAMO, 2022).
While numerous studies have explored treatment assignment strategies for a given sample, there has been limited discussion on the initial selection of a sample from a heterogeneous population. To address these issues, we study how various decision-theoretic frameworks, including minimax regret, utility maximization and cooperative bargaining, can be used to guide sample selection in randomized experiments. We consider a model where different subpopulations may differentially benefit from the knowledge gained in the study, and study participation may involve burdens or rewards which may also manifest themselves differentially among groups. We illustrate how different beliefs and objectives can lead to notably different sample allocations.