Tool / Dataset2026
Tool / Dataset 92A guarded LLM tutor reached its withholding targets in scripted tests, but student learning remains unmeasured
Yusuf Pisan
arXiv preprint
Pisan reports a deployed programming tutor that places an eight-rung help ceiling outside the generating LLM, strips solution code deterministically and judges risky replies against a per-turn contract. Across roughly two dozen scripted turns per calibration run, earnest-reply revisions fell from 43% to 0% and audited ceiling compliance rose from about 77% to 100% after measurement and policy defects were repaired. The evaluation used synthetic personas, not students, so it establishes contract compliance rather than durable learning.
Socratic tutoringanswer withholdingprogramming education
Read 500-word summary →