retrieval-augmented generation

Lecture-video curation grounded AI help in course material, but the pilot measured engagement rather than learning

Owen Tang, Alexandra Vassar, Jake Renzella

arXiv preprint

Tang, Vassar and Renzella tested an alternative to open-ended AI answers: use LLMs to retrieve short, educator-delivered lecture clips for novice programming questions. Proprietary models produced relevant and sufficient selections across five benchmark queries, and a 903-student pilot showed repeat use and positive voluntary ratings. However, low overlap with one lecturer, LLM-only quality judgments and no learning-outcome measure mean the study demonstrates retrieval feasibility and engagement, not safer or better learning.