Generating In-Context, Personalized Feedback for Intelligent Tutors with Large Language Models

Reddig, Arora, and MacLellan study a question that sits directly at the current boundary between classic intelligent tutoring systems and generative AI: can a large language model generate useful, in-context feedback for learners inside an ITS? The paper focuses on GPT-4 and the Apprentice Tutor College Algebra ITS. Instead of asking whether a chatbot can generally answer math questions, the authors ground the model in tutor context and student error data. That makes the article especially valuable for AIEDHK because it treats generative AI as a component inside a structured tutoring system rather than as a stand-alone teacher.

The paper examines three linked tasks. First, the LLM needs to diagnose student errors. Second, it needs to generate corrective feedback that responds to the specific error. Third, the system needs a way to assess whether the diagnosis and feedback are accurate and helpful. This workflow matters because feedback generation is one of the most tempting applications of LLMs in education. Teachers and tutor authors spend substantial time crafting hints, explanations, and bug-specific messages. If an LLM could reliably produce such feedback, it might reduce authoring burden and make tutoring systems more responsive to unusual learner responses.

The results are deliberately mixed. The study reports that GPT-4 can diagnose a range of student errors, but performance drops when responses are more complex or contain multiple problems. It also finds that generated feedback is often relevant and specific, yet a substantial share of hints are too general, incorrect, or reveal the answer. The authors also test whether an LLM can help evaluate generated feedback. That automated quality-control path is promising, but the reported helpfulness pass rate is low enough to show that autonomous feedback pipelines are not ready to be trusted without stronger review.

This makes the paper useful precisely because it resists a simple pro-AI or anti-AI conclusion. It shows that LLMs can add flexibility to ITS feedback, especially when they are given structured context from the tutor. At the same time, it documents the risk of misleading hints, overhelping, and weak automated evaluation. For research translation, the key point is that feedback quality is a safety issue. A fluent hint can still harm learning if it points to the wrong rule, masks a misconception, or gives away the solution before the learner has done the reasoning.

For AIEDHK, the article can become a practical checklist for LLM tutor design. A trustworthy system should specify the tutor context supplied to the model, the student error categories it can diagnose, the threshold for showing feedback, the human or automated review process, and the policy for withholding low-confidence hints. The paper also supports a hybrid design direction for Hong Kong products: combine the curriculum structure and learner modeling of ITS with the language flexibility of LLMs, while keeping teacher oversight and evidence-based quality checks visible. The strongest message is not that LLMs replace tutor authoring, but that they may extend it when embedded in a disciplined tutoring architecture.