Primary Submission Category: Design of Experiments
Designing Experiments to Evaluate Algorithm-Assisted Decision-Making in the Presence of Behavior Adaptation
Authors: Maggie Wang, Michael Baiocchi,
Presenting Author: Maggie Wang*
Algorithm-based decision support tools have the potential to enhance decision quality in a range of applications, including in medicine and in criminal justice. Algorithms are commonly evaluated on retrospective data using performance metrics like accuracy, but these metrics do not necessarily reflect how an algorithm-based tool would change decision-making if it were integrated into a workflow where the human user has the final say. Not only might decision-makers be overly trustful of or averse to tool output, but decision-making behavior may actually adapt with repeated exposure to the tool, e.g. due to gradual building of trust. While randomized experiments can provide better evidence on the impact of decision support tools than retrospective studies, traditional experimental designs and effect estimands still may not correctly capture the impact of the tool because they ignore behavior adaptation. If unaccounted for, behavior adaptation can lead to biased estimates of tool effectiveness and incorrect conclusions about whether the tool should be deployed. In this work, we define time-varying effect estimands that either account for or explicitly measure the impact of behavior adaptation by contrasting different sequences of tool exposure. We then propose experimental designs that target these effect estimands. We demonstrate the utility of our designs with simulations based on a real-world pilot study of a clinical decision support tool that predicts patient deterioration.