pedagogical guardrails

Unguarded GPT raised assisted practice scores but reduced unaided mathematics performance in a high-school RCT

Hamsa Bastani, Osbert Bastani, Alp Sungu, Haosen Ge, Özge Kabakcı, Rei Mariman

Proceedings of the National Academy of Sciences

In a preregistered cluster RCT with nearly 1,000 Turkish high-school students, a standard GPT-4 interface raised assisted practice performance by 48% but reduced later unaided exam performance by 17%. A teacher-informed tutor raised practice performance by 127% and largely removed the penalty without creating a positive exam gain.