Primary Submission Category: Machine Learning and Causal Inference
Advancing Real-World Evidence Studies with Multi-Agent LLM Systems for Causal Inference
Authors: Rachael Phillips, Tianyue Zhou, Mark van der Laan,
Presenting Author: Rachael Phillips*
Large language models (LLMs) offer the potential to revolutionize the design and analysis of real-world evidence (RWE) studies, which leverage data from sources such as electronic health records, claims databases, and registries to inform clinical and regulatory decisions. Intuitive interfaces that engage users in natural language interactions, LLMs provide contextualized assistance and can elucidate trade-offs among different choices. In this work, we introduce a novel multi-agent LLM system tailored for RWE studies that is comprehensively grounded in causal inference frameworks and aligned with regulatory guidelines. We present applications where our customized co-pilot assists researchers in critical steps such as handling intercurrent events, defining interventions, and assessing causal identifiability. Comparative results from real-world examples illustrate the superiority of our co-pilot over proprietary solutions like ChatGPT, which often conflate critical study elements and muddle interpretations. By integrating domain-specific expertise and collaborative agent interactions, our system represents a step change in leveraging LLMs for causal inference. This presentation will conclude with a discussion of future directions and challenges, including ensuring responsible use, benchmarking performance, and fostering interdisciplinary collaboration to establish trust and governance in these AI-assisted methods.