Primary Submission Category: Applications in Physical Sciences, Engineering, Environment and Miscellaneous Applications
Impact of Metacognitive Conversations In Large Scale LLM based Intelligent Tutoring System
Authors: Duy Pham, Kirk Vanacore, Zeyneb Sena Sarioglan, Owen Henkel, Ryan Baker,
Presenting Author: Duy Pham*
In an effort to revolutionize math education in underserved regions, our project leverages advanced AI technology to enhance both student engagement and performance in Rori, an open-source Intelligent Tutoring system. Rori is an AI-powered math tutor accessible via WhatsApp that delivers over 500 micro-lessons based on the Global Proficiency Framework, providing scaffolded practice questions and interactive feedback through natural language processing. This project aims to enhance student engagement and academic performance on Rori by integrating two LLM-based metacognitive conversational modules—one that normalizes making mistakes and another that promotes a growth mindset—and by employing propensity score matching to balance key covariates such as student age, the level of difficulty of content they start in the platform, and their reasons for using Rori, thereby isolating the causal impact of these interventions as we track usage metrics and math performance outcomes.