Preprint: A tutoring-loop review separates convincing dialogue from evidence of learning

A preprint by Sayantan Adak and colleagues, revised on October 14, 2026, organizes conversational AI tutoring around the connections between learner information, instructional action, training and evaluation. Its audit covers 88 papers. The authors find that evaluating pedagogical quality is considerably more common than demonstrating a connected adaptive pathway. They distinguish immediate performance from learning, later retention and transfer. The review searches through August 31, does not pool learning effects, and concentrates on conversational systems. Its categories simplify varied studies, so it should be read as an evidence map rather than a universal effectiveness estimate.

For AIEDHK, a useful next step is to turn that distinction into a small lesson-review activity. The following example is an editorial proposal, not an intervention evaluated by the review. A mathematics teacher could begin with one learner explanation and ask colleagues to identify exactly what it reveals. A correct final answer might coexist with an uncertain method. An incorrect answer might reflect a calculation slip rather than a misunderstanding. Those possibilities call for different questions before any instructional response is selected.

The team could prepare two fictional conversations about proportional reasoning. In one, the learner understands the relationship but miscalculates. In the other, the learner applies addition where multiplication is needed. Reviewers would inspect the assistant's next question and explain how it responds to the particular evidence. If the same generic hint appears in both conversations, the team would have a concrete reason to investigate the supposed personalization more closely.

A classroom pilot could then keep a short record of the learner's initial reasoning, the help offered and the next independent explanation. The teacher would decide in advance what change would count as useful progress. For example, the learner might correctly explain why two quantities vary together in a different situation. Completing another nearly identical exercise with the assistant still present would answer a narrower question. Keeping these outcomes separate would make the eventual report easier for colleagues to interpret.

Students could participate in the review by describing which part of the assistance they used and what they still find uncertain. They might revise an explanation in their preferred teaching language, draw a representation, or demonstrate a method aloud. These options would help the teacher inspect reasoning without making polished English the only available form of evidence. Any resulting differences should be investigated in the local setting rather than assumed from the interface.

The same record can support practical decisions about workload. A teaching team could note the time spent interpreting unclear exchanges, correcting unsuitable hints and selecting the next task. It could retain a few unsuccessful examples alongside promising ones. Such a pilot would give an institution a more informative starting point than counting messages or completed exercises alone. The proposed aim is modest: establish which visible learner evidence justifies continuing, revising or stopping a particular tutoring activity, then decide whether a broader evaluation is warranted.