Tutor CoPilot improved short-term topic mastery by supporting human tutors, with the largest gain among lower-rated tutors

Wang and colleagues test a human-AI design that supports tutors during instruction rather than placing a chatbot in front of students. Tutor CoPilot reads the current mathematics problem and tutor-student conversation, then suggests guidance such as a probing question, explanation, hint or similar problem. The human tutor decides whether to use or edit the suggestion. The preregistered tutor-level randomized trial was conducted with a virtual tutoring provider and nine Title I schools in one southern U.S. district.

At launch, 782 tutors were active, with 386 assigned access and 396 in control. They served 1,787 students in Grades 3 through 8 across 4,136 mathematics sessions over two months, producing more than 550,000 messages. The primary causal outcome was whether a student mastered the lesson topic on the provider's exit ticket. Access to Tutor CoPilot increased that mastery rate by four percentage points in the intention-to-treat analysis. For students taught by tutors who had lower prior quality ratings, the gain was nine percentage points.

Conversation analysis suggests a mechanism. Tutors with access used more guiding questions, asked learners to explain reasoning and gave fewer direct answers. The system may therefore distribute elements of expert tutoring practice at the moment they are needed. At the study's usage level, the authors estimate language-model cost at about 20 dollars per tutor per year. That figure excludes product development, training, monitoring, privacy, support and wider deployment costs.

The longer-term result is essential: the study did not find a statistically significant improvement on the end-of-year mathematics test. The exit ticket is a meaningful proximal measure, but it cannot establish durable achievement or transfer. The work also took place with one provider and district over two months, with variable tool use. Some suggestions were not grade-appropriate, and de-identifying names cannot prevent learners from revealing other personal details in free conversation.

For implementation, the human-in-the-loop architecture is promising only if the human remains capable and accountable. Tutors need training to reject a plausible but poor suggestion, adapt language to the learner and protect personal information. Providers can evaluate suggestion quality by topic, grade and tutor experience, sample interactions for human review and show tutors why a recommended move may help. A system that silently optimizes acceptance rates could undermine that judgment.

For Hong Kong tutoring and school support, a pilot should test local curricula, Cantonese and English dialogue and varied learner needs. Outcomes can include immediate mastery, delayed unaided problems, tutor practice and equity across tutor experience. Full cost should include onboarding and review.

The paper's strongest finding is appropriately narrow: real-time AI support improved short-term topic mastery and especially helped learners served by lower-rated tutors. Whether that support produces sustained learning remains unproven, making delayed independent assessment a necessary next step.

The human tutor should also be able to report a poor suggestion without interrupting the lesson, and programme leaders should study rejection as useful evidence rather than failure. Good oversight learns from the moments when professional judgment overrides the model.