human review

AI scribes produced fuller educational feedback notes, but human editing did not remove every error

Jaideep S. Talwalkar, David Chartash, Lisa Zhang, Michael Makutonin, Conrad W. Safranek, Anne Elizabeth Sidamon-Eristoff, Lee H. Schwamm, Donald S. Wright

JMIR Medical Education

A randomized workshop study assigned 13 medical educators to human-only or AI-assisted feedback documentation. AI-assisted narratives scored higher for inclusion of useful feedback elements, but they were much longer and sometimes inaccurate. Review of 117 generated elements found eight mischaracterizations and two hallucinations; editing reduced most mischaracterizations but did not remove the two hallucinations. The study measured documentation quality and workload, not subsequent student competence or clinical outcomes.