Primary Submission Category: Causal Inference and Bias/Discrimination
Data-Adaptive Experimentation to Find Contexts with the Most and Least Discrimination
Authors: Jennah Gosciak, Daniel Molitor, Ian Lundberg,
Presenting Author: Jennah Gosciak*
Randomized experiments reveal discriminatory choices. From audit studies to online experiments, designs follow a common recipe: present a decision-maker with a profile that holds constant the context (e.g., the political experience of a candidate) and randomizes a signal (e.g., the age of the candidate). This design isolates the causal effect of the signal on the choice: the decision must be caused by age. But what if discrimination differs across contexts? Voters might prefer a younger candidate only if that candidate has political experience, for example. Experiments that fix different contexts would detect different amounts of discrimination. Standard designs cannot discover this variation because the context is fixed. Conjoint experiments cannot discover this variation because they are powered for average marginal effects. By applying Thompson sampling methods, we data-adaptively discover contexts with high and low levels of discrimination. We illustrate with two new pre-registered online experiments. Our first experiment (completed) explores how the effect of candidate age on voter preferences depends on the context of candidate race, gender, and experience. Our second experiment (funded and IRB approved) explores how hiring discrimination against mothers may depend on the context of the applicant’s race and educational credentials. Our goal is to show how experiments can not only detect discrimination, but adaptively discover the contexts where it is most severe.