Primary Submission Category: Machine Learning and Causal Inference
Integrating Detection and Causal Models using Fully Latent Principal Stratification
Authors: Kirk Vanacore, Adam Sales,
Presenting Author: Kirk Vanacore*
Predictive models can provide real-time estimations of human attributes, dispositions, propensities, or attitudes. When implemented in computer applications, they can provide insight into users’ latent states. However, they do not address the optimal actions these systems should take given the predictions. Fully Latent Principal Stratification (FLPS) provides one solution by allowing for the estimation of subgroup effects when those subgroups are determined after randomization and defined by a latent variable. The integration of detection models and FLPS have the potential to turn predictions into actionable recommendations based on estimated causal effects.
We illustrate this application using an example study of a detection model implementation in a Computer-Based Learning Platform (CBLP). In this example we address the issue of ‘gaming the system’ – a behavior categorized by attempting to progress through a learning activity without learning – in a CBLPs. Using the combination of a ‘gaming the system’ detection model and FLPS, we are able to estimate how students who have a high propensity to game the system in one CBLP would respond to modifications of the original CBLP as well as CBLPs with different pedagogical approaches. Through this analysis, we determine that simple manipulations of feedback systems within a CBLP may be more optimal than global changes, such as gamification.