Preprint: Adaptive tutor prompts changed interaction patterns without establishing better learning

Po-Chin Chang and colleagues describe an AI tutor that selects among instructional prompts using subject information and feedback on dialogue. The June 18, 2026 preprint version reports an online comparison involving 656 conversations from 359 Dutch high-school students. Some longer conversations became shorter under adaptive prompting. However, combined exercise uptake was 20.8 percent with the router and 19.6 percent with the baseline, a difference that was not statistically significant. The limited exercise-accuracy analyses also did not establish an advantage. These findings concern interaction patterns and preliminary evaluation, rather than demonstrated improvement in learning.

The evaluation relied partly on AI-generated pedagogical scores. Many learners did not proceed to exercises, and some subjects had very little data. The authors identify these limitations when discussing learning outcomes and adaptation across subjects. This report uses version 1, which was available before the archive coverage date; it does not import the later revision's publication status.

For educators, a useful question is what a shorter conversation actually means. A student might reach a helpful next step more quickly, or might leave before resolving a difficulty. Consider a proposed classroom trial in which a teacher compares two approved styles of hint for the same geometry task. One asks the learner to explain a relationship; another points to a particular feature of the diagram. Students would still have to complete and explain the solution themselves. This example is an editorial proposal, not a procedure evaluated in the paper.

The trial could retain the original problem, the hint, the student's next attempt and the explanation for that attempt. Those records would let the teacher inspect whether the hint directed attention toward the relevant mathematical relationship. Counting messages alone would leave that question unanswered. A concise exchange followed by a correct explanation could be useful; an equally concise exchange followed by an unsupported answer would require a different response.

Teachers could also ask students to judge the help they received. A short reflection might identify which part of a hint was useful, what remained confusing and what they would try without assistance. These judgments should be compared with the student's work, because feeling helped and demonstrating understanding answer different questions. When the two disagree, the teacher would have a concrete example for discussing how to use feedback thoughtfully.

A local evaluation should define its main learning outcome before choosing a winning prompt. It could compare performance on a new, unaided problem and retain a later check to examine whether the reasoning persists. Participation and missing exercises should be reported alongside the results. If learners who find the tutor confusing are also less likely to complete the assessment, a simple average among completers may give an incomplete picture.

The practical design choice is therefore broader than selecting the prompt with the highest interaction score. A school needs a way to connect a tutoring decision with what the learner subsequently understands and can do. Adaptive prompting offers a candidate mechanism for making support responsive, while the educational case still depends on evidence from the intended learners, subjects and assessment conditions.