Primary Submission Category: Regression Discontinuity
An application of regression discontinuity design for evaluating the impact of program features in a computer-based learning platform
Authors: Kirk Vanacore, Adam Sales, Ben Hansen,
Presenting Author: Kirk Vanacore*
Regression Discontinuity Design (RDD) is commonly employed in economics, public policy, and education research, but it is underutilized in human-computer interaction research. Yet, features in computer programs are often good candidates for RDD, because many decisions in these programs are made based on a cut point. In the area of computer-based learning platforms (CBLPs), some examples include the administering of rewards, prescribing usage recommendations, and determining whether students have mastered a knowledge component. These mechanisms, which determine how students experience learning programs, provide opportunities for understanding the impacts of these features on users’ behaviors and outcomes.
In the current study, we evaluated the impact of game-based failure on students’ persistence behavior in a gamified CBLP using a regression discontinuity design. We found that game-based failure increases the likelihood of students engaging in productive persistence as they play an online gamified algebra program. This finding suggests that gamification features in learning programs help sidestep the negative aspects of failure and leverage those failure experiences for learning. This work also illustrates the usefulness of regression discontinuity designs in evaluating the impact of features in online learning games to provide insight into causal mechanisms through which program features influence students’ learning processes.